The single point of failure setup with Purelymail does indeed give me the occasional worry. But fortunately there are a lot of standard tools and processes for migrating email content from one server to another via IMAP and email archives. You can e.g. have a Thunderbird client running at all times, connected to your Purelymail server via IMAP, that will download an offline copy of everything you have in your Purelymail account(s). If Purelymail goes down tomorrow, you can find another provider that supports IMAP, update your DNS MX records, upload your Thunderbird archive to the new provider, and start using your new email provider as never even happened.
Email migrations from hosting provider to another have been among the easiest migrations I’ve ever had to work with. You only have to convince both parties to open up an IMAP server interface for you. ProtonMail can do this with Proton Bridge, but they also have a tool for importing raw .mbox files from Thunderbird.
Yes, this is one of the wonderful benefits of federation. Thanks to IMAP, you can constantly keep an offline copy of all your emails. If Purelymail stops working for whatever reason, you can just sign up somewhere else, point your domain MX records to the new server and upload your emails there via IMAP or their own import tool. Migrating takes less than an hour, and even that is only because it takes a while for the DNS MX records to propagate.