West and Central Africa are subject to various hydrological regimes that make the region very unique.

The hydrological regime of a river at a given point is the average behaviour of its flows at this point during one hydrological cycle (the form of its hydrograph). In West and Central Africa, two families of hydrological regimes exist (Rodier, 1964): tropical and equatorial regimes. The tropical regime is characterized by one season of high flows and one season of low flows. The equatorial one is characterized by two seasons of high flows and two seasons of low flows. Depending in particular on the duration of each of these seasons and the extent of the high or low flows, we can distinguish desertic, subdesertic, Sahelian, transitional, and pure tropical hydrological regimes, and boreal transitional, austral transitional, and pure equatorial hydrological regimes. Fig. 1a provides an overview of the spatial location of these different hydrological regimes derived from Rodier (1964) in the study area.

The study focuses on 241 catchments in West and Central Africa (Fig. 1b); these catchments encompass the main rivers of the region and the Sahelian regime, pure tropical, transitional, and Dahomean transitional tropical regimes, and boreal transitional and pure equatorial hydrological regimes.

Monthly runoff chronological series come from SIEREM database managed by HydroSciences Montpellier (http://www.hydrosciences.fr/sierem/; Boyer et al., 2006). The data were checked for errors.

Monthly rainfall (P) and potential evapotranspiration (PET) continuous time series come from CRU TS 4.00 grids (Harris, 2017). The data are observational products. The spatial resolution of the grids is 0.5 × 0.5 degree and the data cover the period from 1901 to 2015.

Fig. 2 showed the diversified hydroclimatic characteristics of the catchments under investigation, notably the precipitation, the PET, and the specific annual runoff. Fig. 2 also reveals the period during which the hydrometric stations function, and the percentage of hydrological data gaps by catchment.

The time period covered for the study is from 1950 to 1990 due to the limited services given by the national hydrological services and the difficulties to get recent hydroclimatic data of African regions.

Due to limitation of data, it is admitted an acceptable percentage of gaps in runoff, according to the hydrological regime of the catchments of interest. For tropical catchments, especially the Sahelian and the pure tropical catchments, the acceptable percentage of gaps has been set at 50%. The gaps recorded are generally in periods of low water levels where the flow is most often perennial. For equatorial catchments, the acceptable gap is set at 30% because these rivers have a continuous flow.

Most of the selected catchments range in size from 200 to 1000 km². They are mostly unregulated with no major storage or irrigation schemes.

Five monthly lumped rainfall-runoff models (RR) with continuous reservoir types were used in this study. In spite of their parsimony (only a few free parameters), they showed a good level of efficiency in previous studies (Ouermi et al., 2015; Paturel, 2014; Paturel et al., 1997) and correspond to several representations of the rainfall–runoff transformation. Within a model, the versions differ due to the number of parameters to calibrate:

• • Two versions of the Water Balance model (Conway, 1997) with two or three parameters (WB2 or WB3), which is a combination of the Thornthwaite water balance approach (Thornthwaite and Mather, 1957) and the large scale hydrological modelling approach used by Vörösmarty et al. (1989) and Vörösmarty and Moore (1991);
• • Two versions of Makhlouf's model (Makhlouf and Michel, 1994) with two or three parameters (MK2 or MK3);
• • One version of Mouelhi's model (Mouelhi, 2003; Mouelhi et al., 2006) with two parameters (MO).

MK and MO models are monthly GR models (https://webgr.irstea.fr) with two reservoirs, a “ground reservoir” and a “transfer reservoir”. WB models are single reservoir models.

The first version of the MK model has four parameters. According to Makhlouf and Michel (1994), two of these parameters can be fixed in specific climatic conditions. They suggested that, in other climatic conditions, it is preferred that all four parameters be calibrated. After various uses of the MK model, Ouédraogo (2001) and Lubès-Niel et al., 2003 found that, for most of the African catchments studied, the parameter linked to direct flow is zero and the parameter linked to soil condition characteristics (A) can be assimilated to the Water Holding Capacity (WHC) of the soil, data mapped by the FAO (1995). Thus, it is proposed in this study that the third parameter, γ in WB and A in MK, be replaced by WHC.

Even if MK and MO are GR models, their structure is a little bit different: the MO model assumes that there is conceptually an exchange of water with outside parts of catchment, a factor which is able to adjust the rainfall inputs to provide higher NSE values in calibration; not in the MK model.

We refer to Supplementary Material 1 for a further overview of the characteristics of the five tested models.

The parameters are calibrated using the method of Rosenbrock followed by the simplex of Nelder and Mead (Servat and Dezetter, 1988). For this, we use three goodness-of-fit criteria (GOFC), ranging from –∞ to 1:

• • NSE(Q) Criterion (Nash and Sutcliffe, 1970),

• • NSE(√Q) Criterion (Nash and Sutcliffe, 1970),

• • KGE Criterion (Gupta et al., 2009),

The NSE criterion (Q) is the most widely used one for evaluating the performance of hydrological models because of its simplicity. However, it gives more importance to high flows than other parts of hydrograph. Its NSE variant (√Q) gives equal consideration to all parts of the hydrograph.

The KGE index is a result of decomposition of NSE(Q) and represents a compromise between three evaluation criteria, correlation coefficient, bias error and standard deviation ratio (Dakhlaoui et al., 2017).

The differential split-sample test (DSST) proposed by Klemes (1986) is the common typical testing procedure to investigate the parameter dependency on climate and the related consequences on model efficiency. This is a specific case of the split-sample test (SST), where calibration and validation periods are chosen according to their climatic differences. Finding limits to the DSST, Coron et al. (2012) proposed a generalized methodology to evaluate the validity of RR models for use under non-stationary climatic conditions: the Generalised-Split-Sample Test (GSST). According to the authors, GSST overcomes the limits to provide comparable results under various conditions and over a wide range of parameter transfer conditions, thus resulting in more robust conclusions on parameter transferability.

The method follows three steps

• • numerous subperiods are created by a sliding window moving by one year according to one climatic characteristic (e.g., mean rainfall or PET for the catchment);
• • a hydrological model is calibrated on each subperiod using previously selected quality criteria; this provides one parameter set per period;
• • for each calibration subperiod, the optimized parameter set is used to perform all the possible validation tests on independent subperiods. Validation subperiods overlapping with the calibration ones are not considered to ensure strict independence of calibration and validation conditions.

Nevertheless, Coron et al. (2012) note that their method is limited because it does not always make it possible to attribute the change in model behaviour to a specific climate or environmental change experienced in a catchment.

In the present study, the length of the chosen sub-period is five years for calibration and validation. Sub-periods of five years permit to maximize the climatic differences between two periods and to test models in much contrasted conditions.

However, the number of sub-periods is different from one catchment to another because of the available runoff data for the considered catchment. Given the available runoff data, 7634 calibration runs were possible in this study.

The goodness-of-fit criterion in the ‘model simulation’ (using an optimized parameter set from model calibration versus a different subperiod) is then compared to the model calibration results to quantify the variation in GOFC. This relative variation measures the transferability of the model to climate-contrasted periods or the robustness of the model. It is formulated as:

The “donor” period is the calibration period and the “receiver” period is the validation period. So Performance_donor is the calibration performance and Performance_receiver is the validation performance.

Some authors, such as Seiller et al. (2012) and Dakhlaoui et al., 2017, criticized this way of expressing model robustness. They found that it underestimates the robustness of the models by exaggerating the performance losses of the models. They found other methods of measuring robustness. Dakhlaoui et al., 2017 proposed:

An overview of calibration performance was provided to evaluate the quality of the sets of reference parameters and to check that the models perform reasonably well in calibration. The calibration performance in terms of GOFC is summarised as boxplots according to models (Fig. 3) and according to criteria (Fig. 4). The horizontal line in the box is the median of GOFC values. The upper and lower envelopes show the 75th and the 25th percentile values and the upper and lower whiskers show the 95th and 5th percentile values, respectively.

An overall analysis of the calibration performance of the models showed that the GR models are more efficient than the WB models. The median and the upper whisker of the plots are lower for WB models than for the GR models. The number of runs with at least 60 for GOFC is higher for GR models than for WB models. The concept of GR is more applicable to the study area investigated, which is probably due to the structure of GR models, which have one routing reservoir instead of none for WB models.

The MK models are slightly more performant than the MO model in calibration. This is confirmed by the number of runs, which is higher for MK models than for the MO model.

The results of calibration performance are more distinct for NSE(√Q) than for the two others GOFC because of the lower whisker, which is higher than for others. We must note that in the case of KGE, the upper whisker is slightly similar to the two other GOFCs.

The different versions of the MK and WB models with two or three parameters are almost equivalent in calibration, despite the different numbers of parameters to calibrate whatever the performance criteria may be. It seems that assimilating the soil reservoir parameter to a water holding capacity of the soil is appropriate. It must be due to the fact that the two models are not very sensitive to this parameter.

More specifically, the analysis of the calibration performance of the models shows also behaviour differences between GOFC components. Fig. 4 shows that GR models are the most efficient in terms of NSE(√Q) and WB models in terms of KGE (confirmed by number of runs). Does that mean that the structure of a model and GOFC are linked? This could be partly explained by the fact that models are generally implemented using a particular GOFC.

Because of highest values for NSE(√Q), the assumption can be made that the structure of GR models is the most capable of reproducing the entire hydrograph than reproducing the highest flows. With higher KGE coefficients compared to NSE, the structure of WB models gives a better representation of the variability of the flows than the flows themselves.

The analysis of calibration performance displays overall similar trends of calibration performances according to the annual precipitation amount, except for the highest ones (Supplementary Material 2). For the WB3 model, the calibration performance increases with the average annual rainfall whatever the GOFC. For WB2 confronted in terms of NSE criteria, performance increases with annual rainfall and falls around the average rainfall of 2600 mm/year. In terms of KGE, the performance increases with annual rainfall. It has been proven above that WB models better reproduce flow variability. The more variable the climatic conditions are over the calibration period, the more difficult it is for the model to reproduce the flows. Wet catchments often display less variability. This could explain WB3's behaviour with rain. The one of WB2 is probably different because of the replacement of the third parameter by WHC_max, a measurable value in the field. For the GR models in whatever NSE criterion, performance increases slightly with annual rainfall, before falling to around 2600 mm/year in rainfall. It is indeed less clear for the MO model. For the KGE criterion, performance drops to around 1000 mm/year in rainfall, from where it rises with rainfall and then falls to around 2600 mm/year. We do not find any explanation for this overall behaviour.

As with annual rainfall, the behaviour of model calibration performance as a function of PET depends on the model and the GOFC (Supplementary Material 3). For the MO model and for all GOFCs, the calibration performance increases with annual PET up to around PET 1800 mm/year and drops sharply to around PET 2200 mm/year. For MK models and in terms of NSE(Q), the same behaviour as that of MO is recorded. In terms of NSE(√Q) and KGE, there is a slight decrease in performance up to around PET 1200 mm/year, which then rises to 1800 mm/year, from which it falls to 2200 mm/year. The WB models show very irregular behaviour with PET. For WB2, the performance decreases with PET in terms of NSE. In terms of NSE(√Q) and KGE, it drops to values of 1400 mm/year and 1600 mm/year. It then rises to the value of 1800 mm/year and 2000 mm/year, from which it falls to 2200 mm/year. As for WB3, it seems to be insensitive to PET because a clear relationship does not emerge between these calibration performances and PET. An explanation for these behaviours is hardly found because the previous tests show a very low sensitivity of the studied models with respect to PET. An analysis of the calibration performance in terms of hydrological regimes of the catchment was carried out against the background of a geographical map. No link was found between the proper calibration of flows and the hydrological regime of the considered catchment.

As described earlier, the GSST procedure is used over five year periods for 241 catchments. Fig. 5 (according to the GOFC) shows the performance loss in the GOFC compared to the model calibrations for all of the catchments and all the periods. The cumulative distribution function of performance loss in the GOFC (y-axis) is plotted versus the percentage difference in the GOFC (x-axis) in the simulation period (receiver period) relative to the calibration period (donor period). The percentage difference is calculated in such a way that the donor period is used as the denominator (Eq. (5)). Negative values on the x-axis represent simulation results where there is a reduction in GOFC compared to calibrated GOFC values and vice-versa.

Fig. 5 shows an interesting point in about 10% (up to 15–20%) of the simulations; the simulated GOFC values can be greater than the calibrated GOFC values. This means that parameters from a donor period transferred over a receiver period lead to higher GOFC values than over the donor period. This is possible if, between the two periods, donor and receiver, the values of GOFCs in calibration are sufficiently different, and in particular if the GOFC in calibration of the donor period is lower than the GOFC in calibration of the receiver period. For a reason that is not necessarily known (data? punctual change over time in the rain-runoff relationship? etc.), the model does not match well for one period, while it matches better for another one.

An overall analysis of models robustness with respect to the performance criterion shows that the GR models are more robust than the WB models, whatever the GOFC. The order of magnitude of robustness is higher, in terms of NSE(√Q), then NSE(Q) and finally KGE.

The robustness of GR models varies according to the chosen GOFC. It is almost the same in terms of NSE(Q) and slightly different in terms of NSE(√Q) and KGE. The MK models have very close robustness which is higher than MO one in terms of NSE or NSE(√Q). On the other hand, in terms of KGE, the MO model is more robust than MK models.

The comparison of the robustness of the versions of MK and WB models shows that the version with three parameters is slightly more robust than the version with two parameters.

The three graphs in Fig. 5 show a break at the percentile 99% of the cumulative frequency graph for all studied models and GOFC and around 20% of transfers gain in performance. No explanation is found for the catchments for which the robustness is high. Differences exist from one model to another and from one criterion to another. But, in general, models are all robust for Central Africa pure equatorial catchments. Very wet watersheds with less variation in hydroclimatic conditions would be easier to model.

When comparing these preliminary results on robustness to those on calibration previously obtained, it appears that there is a link between the chosen GOFC and a model. The GR models have better calibration and robustness performances with NSE(√Q) compared to the WB models that have better performances with KGE.

In terms of rainfall, a link can be established between model robustness (performance loss) and the difference in mean rainfall between calibration and simulation periods (Supplementary Material 4). The behaviour is different depending on the GOFC and the model chosen. Nonetheless, for all models and selected GOFCs, the minimum of performance loss (median) is for a relative change in P of ±5%: the receiver period is relatively 5% wetter or drier than the donor period. In the same way, the performance loss in GOFC values is not symmetric for positive and negative relative changes in mean rainfall. The performance loss in GOFC values is generally greater when a model calibrated over a wet period is used to model runoff over a drier period compared to when a model calibrated over a dry period is used to model runoff over a wetter period. For WB models and regardless of the chosen GOFC, as expected, the performance-loosing GOFC values are generally higher for a larger difference between the rainfall in the calibration (donor) and simulation (receiver) periods. For the GR models, it was observed a particular behaviour since there is a slight inflexion point for the positive ΔP (the receiver period is relatively wetter than the donor period), at ΔP = +30%: the median of the performance losses at ΔP = +50% corresponds to a less significant loss of performance than for ΔP = +40%. This observation should perhaps be put into perspective since the number of runs that could be made corresponding to ΔP = +50% is much less important than the number of runs corresponding to ΔP = +40%. However, this observation is common to all GOFCs, which still gives it some weight. The analysis undertaken in this study does not provide any explanation for these results and more analysis is required to investigate the details of the calibrated parameter values and link them to the model structure.

The range of performance losses during parameters transfers depends on the chosen model and the GOFC. However, it is minimal for a ΔP = 0%. For NSE(Q), it varies overall as the median. For KGE, the range remains significant, regardless of the observed rainfall change.

Therefore, it appears that in terms of median of performance losses by class of ΔP, but also in terms of range, it is better to calibrate the models on a dry period than on a wet period. The NSE criteria, and in particular the NSE(√Q) criterion, also allow a higher robustness of the GR models. The GR models have very similar robustness performances.

To have an overview of the level of error obtained when parameters are transferred under similar rainfall conditions, averaging the box plots corresponding to −10% and +10% can be done. With NSE(√Q), the median of performance loss is inferior to 10%; slightly higher with NSE(Q) and around 15–20% with KGE. With NSE(√Q), the range is then in the order of ±20% on either side of the median. Calibrating GR models with NSE(√Q) on a dry period and applying them on a wetter period will lead in 75% of cases to a loss of performance that will be inferior to 15–20% at most. This can be considered acceptable if the calibration is considered of good quality.

The analysis of the robustness of the model to PET variation shows much more consistent results in terms of models and chosen criteria (Supplementary Material 5). The variations observed in terms of PET are much lower than those of rainfall and are only in the order of a few percent in the study area: −6.5% to +6.5%. However, the magnitude of the performance losses is of the same order of magnitude as that obtained with a change in rainfall. The WB models are much less robust than the GR models. NSE criteria also give better robustness qualities. As for rainfall, the performance loss in GOFC values is not symmetric for positive and negative relative changes in mean rainfall. However, the dissymmetry to change in mean PET is opposite to change in mean rainfall. The loss of performance is lower when the parameters are transferred from a period with higher PET than in the opposite case. The minimum loss of performance corresponds to a ΔPET = 0%. As with rainfall, the NSE(√Q) leads to the best robustness of GR models. In a range from ΔPET = −1% to +1%, the loss of performance is in 75% of cases inferior to 15% for MK, 20% for MO.

Using Eq. (5) of Dakhlaoui et al., 2017, robustness analysis led to results similar to those obtained with conventional robustness calculation. The remark made by the authors that the conventional method would exaggerate the performance losses is not verified in this case.

Fig. 6 is a set of maps giving a spatial representation of the robustness of the different models according to GOFC. These maps were cross-referenced with those of hydrological regimes to also allow an analysis of the robustness of the models in relation to hydrological regimes. For each GOFC and for each calibration/validation run (all periods combined), the number of times a model is the most robust has been counted, and the one with the highest number has been selected.

Fig. 6 shows that in terms of NSE(Q) and NSE(√Q), the MK3 model is slightly more robust than MK2s, whatever the hydrological regime of the catchment may be. In terms of KGE, the MO model seems to be most robust for the West African catchments of pure equatorial and equatorial boreal transitional regimes and for transitional tropical regime Dahomean catchments. For other hydrological regimes, the MK2 and MK3 models are equivalent.