This was one of those frustrating things.
root@mxg:~# update-locale
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_All = "C.UTF-8",
LANG = "en_GB.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
I fixed it with this:
export LANGUAGE=en_US.UTF-8; export LANG=en_US.UTF-8; export LC_ALL=en_US.UTF-8; locale-gen en_US.UTF-8
After that, reconfigure dpkg
root@mxg:~# dpkg-reconfigure locales
Generating locales (this might take a while)...
en_US.UTF-8... done
Generation complete.
