Technical Abstract: Landscape changes complicate assessing improvements in water quality due to conservation-based agricultural best management practices (BMPs) such as cattle crossings and buffer strips. Water quality was monitored before and after BMP installation in the Spring Creek Watershed (Centre Co. PA, Chesapeake Bay Watershed). Conservation BMPs were applied to 91% and 61% of the agricultural streambank within two subwatersheds (BMP1 and BMP2). A forested reference subwatershed (REF) remained unmodified. Land use determined from 2006 digital photography was 30% agriculture, 40% forested, and 30% developed. The effects of BMPs on water quality were evaluated using a hierarchical generalized linear modeling approach. Potential causal factors (land use, area of drainage, soil type, and presence of BMPs) that provided the best fitting models to the water quality response variables (N, P, pH, temperature, total suspended solids, and storm discharge) were determined at multiple scales: watershed, subwatershed, and reach. BMPs did not predict nutrient concentrations despite reducing overall sediment. Drainage area at the stream reach and the subwatershed scales was significantly related to water quality. Land use and soil type were uninformative. Evaluating water quality at the watershed scale requires correction for emergent effects of land use change.