MySQL character sets and storing "high" characters

This is the text of an email sent to one of Fastmail's internal engineering lists, and has been sanitised slightly to remove private information and internal links. Since 1999, Fastmail has used MySQL as its internal database. In those days it was famous for its lack of robustness, and over the years Fastmail has built up a layer of workarounds to correct for this. In 2021 I began a project to upgrade from MySQL 5.1 (2008!) to MySQL 8. MySQL these days is good enough that most of the workarounds were now in the way, and so had to be carefully removed or worked around. This is the story of just one of them.