Mapping the dielectric properties of cells with nanoscale spatial resolution can be an important tool in nanomedicine and nanotoxicity analysis, which can complement structural and mechanical nanoscale measurements. Recently we have shown that dielectric constant maps can be obtained on dried fixed cells in air environment by means of scanning dielectric force volume microscopy. Here, we demonstrate that such measurements can also be performed in the much more challenging case of fixed cells in liquid environment. Performing the measurements in liquid media contributes to preserve better the structure of the fixed cells, while also enabling accessing the local dielectric properties under fully hydrated conditions. The results shown in this work pave the way to address the nanoscale dielectric imaging of living cells, for which still further developments are required, as discussed here.