peer review report 1 on scale dependence of controls on groundwater vulnerability in the water energy food nexus california coastal basin aquifer system

Journal of Hydrology: Regional Studies (2017) 57–58 Contents lists available at ScienceDirect Journal of Hydrology: Regional Studies journal homepage: www.elsevier.com/locate/ejrh Peer Review Report Peer review report on “Scale dependence of controls on groundwater vulnerability in the water-energy-food nexus, California Coastal Basin aquifer system” Original Submission 1.1 Recommendation Major Revision Comments to Author: The authors developed logistic regression models for groundwater nitrate in the California Coastal Basins (regional and three subregions) They identified dissolved oxygen as a significant variable in all of the models and also discussed the influence of other predictor variables They identified “scale invariant” variables, but in my opinion the concept could use clarification The authors said that past models were poor and perhaps overfitted and that “better calibrated models with higher degree of predictive ability” were the primary objective However the concept of calibration vs prediction is not well articulated, and the validation analysis aimed at “predictive ability” is flawed and should be dropped or redone The following comments correspond to line numbers in the manuscript L62: change “is” to “are” L62–76: “Water-Energy-Food-Nexus” is mentioned four times here Consider abbreviating it ă you mean poor fit to validation data? What fit metrics were used? L98–99: Define what you mean by “poor.Do L100: In light of the previous sentence’s reference to over-fitting, it’s not clear what the advantage of better calibrated models is To increase predictive performance, shouldn’t model fit to calibration data be relaxed somewhat to avoid overfitting? Or you want better fit to calibration data so as to better explain the predictor variables and processes involved? Is the model ultimately to be used for prediction or to identify factors important to nitrate? L104–106: It’s not clear how “scale dependence” of explanatory variables is being quantified or what is meant by it in the context of aquifer vulnerability modeling Did you consider a more formal analysis of spatial dependence of explanatory variables, such as semivariogram modeling to obtain correlation length? L107–108: “Better calibration” and “high degree of predictive ability” usually are at odds In general the more tightly calibrated a model is, the less well it generalizes to new data (prediction) This can be seen by comparing different levels of model complexity against validation data L114–122: These statements don’t seem relevant to a research paper L160–161: This would remove temporal variability of nitrate at the well, but there could still be temporal variability over the region What is the range of well sampling dates? DOI of the original article:http://dx.doi.org/10.1016/j.ejrh.2016.01.002 2214-5818/$ – see front matter http://dx.doi.org/10.1016/j.ejrh.2016.11.043 58 Peer Review Report / Journal of Hydrology: Regional Studies (2017) 57–58 Groundwater-quality data section on p 8: As per the above, well sampling dates should be given about here I saw no mention of them in the paper or the supplementary material Without them it is it is difficult to evaluate the appropriateness of the years of land use and fertilizer data used in the modeling L180–183? Would expect a more locally informed estimate of background nitrate in the region, vs using national numbers What is background for this part of CA? Seems that such data would be available given CA’s comprehensive Groundwater Ambient Monitoring and Assessment program Indicate about here that this is the logistic regression modeling threshold L202–205: Mention dates of land use here so the reader doesn’t have to go to the supplemental material Why not use CA DWR land use data instead of a national land cover data set? How does the land use year compare with well sampling dates − is there a sufficient temporal offset to account for system lag? L225: Because LR is widely used,ăsuggest including more references here L242: Mention here that “univariate relations” refers to univariate logistic regression models L254–256: Need some description of the LR fit method used Forward stepwise, backward, best subsets, or .? ă all fertilizer contains nitrate L276: Suggest changing NO3 to “nitrogen.Not L278: Change “slow” to “slows” L283–285: It would be better to test for multicollinearity directly in a multivariate model rather than assume it is not occurring based on univariate correlations See Menard (2001, Applied Logistic Regression, Sage Publications) for use of the tolerance statistic in a logistic regression context L309–325: A plausible explanation is provided for negative relations between N source variables and groundwater nitrate in the central subregion Travel time from the land surface to the well is an important factor in this regard, but there is not a lot to go on as per the above comments on well sampling date In model b, DO is basically just a proxy for nitrate As the authors state on L305-306, the relation between DO and nitrate is well documented Without a source term, this model is not a vulnerability model Did you consider interactions among explanatory variables to try and tease out more processes? L340–342: See above comment on testing for multicollinearity in a logistic regression model L353–354: Could mention the importance of groundwater age about here as well Although all of these samples contain some modern groundwater according to the tritium data, the age mixtures could vary a lot depending on well depth, screen length, and pumping rate L356–357: Not quite sure what is meant by “scale invariant” for DO Because it is present in four of the five models? L363–365: Sampling for Fe and Mn along with DO would provide even more understanding of ambient redox conditions, as per the McMahon and Chapelle (2008) paper L373: Explain what AWC represents about here L376–379: Farm fertilizer is in the regional model and north model, but not the central and south models Why is it ă considered scale invariant?This variable is said to be “least significant” in the regional model, but soil AWC has a higher p value according to Table L391–396: Not clear what “methods” you are referring to You simply assumed the same DO values everywhere to make the maps, correct? How the selected DO values relate to redox status as described by McMahon and Chapelle? L411–413: Predicted nitrate probabilities seem rather low and there is not a lot of difference between the two maps Maybe you could make a map of the differences “Predictive ability” section starting on L416: This is not model validation, this is recalibration of a model to different data By definition, for it to be predictive the observations can’t have been used in model calibration The authors imply that simulated values can’t be compared with observed because the former are in probability units, but in fact it is straightforward to compare simulated and observed occurrences and non-occurrences (1 s and 0s); see for example “Classifying New Observations” in “Logistic Regression Examples Using the SAS System” (1997, SAS Institute Inc.) Or, they could compute deviance values for the validation wells based on the predicted probabilities and observed s and 1s This entire section should be revised or dropped L450: What is the time period? L462–463: Farm fertilizer is described as “scale invariant” but is missing from the central and south models Anonymous ... mention of them in the paper or the supplementary material Without them it is it is difficult to evaluate the appropriateness of the years of land use and fertilizer data used in the modeling L180? ?18 3?... comment on testing for multicollinearity in a logistic regression model L353–354: Could mention the importance of groundwater age about here as well Although all of these samples contain some... occurrences and non-occurrences (1 s and 0s); see for example “Classifying New Observations” in “Logistic Regression Examples Using the SAS System? ?? (19 97, SAS Institute Inc.) Or, they could compute