Cloudflare is a reverse proxy service that protects hundreds of thousands of websites, The Orbit included, from attacks like DDoS, spam, brute force, and various other exploits. Without it, in the adversarial environment that the Internet happens to be for social justice oriented folks, we would be crushed under the weight of people desiring to silence us. So, they’re doing us a great service, and we are indebted to them.

HOWEVER.

Yesterday, after a major vulnerability was discovered in the Cloudflare plugin for WordPress, which could allow sites to be cross-site scripted (a method that might allow you to inject bad code into a site “from the side”), it seems as though they panicked and decided to encode *all* POST and GET data, which caused a major set of problems. People trying to edit posts found every non-alphanumeric character turned into an HTML entity (“:” instead of “:” for instance). Then those entities were being reencoded again (“:”).

Over and over and on and on, the posts were getting more and more corrupted. And that wasn’t the only thing that was busted — admins were being told they didn’t have permissions to access certain pages, because the links to those pages were having parts of themselves converted to HTML entities as well. End users could see the site, but admins were fully hamstrung. Greta was working on Steven Universe episode 8 and got stopped short, emailed me to find out what broke, and to my horror, the auto-updated plugin for Cloudflare was actually hampering my ability to do anything in the WordPress admin. I thought we were in serious trouble, but I tracked it back to the plugin which had just updated to version 1.3.21. I pulled out an older version from Sunday’s backups, 1.3.20, and the problem was resolved. Then I found out WHY they’d updated it, and apparently there are such hacks in the wild right now.

So. Rather than risk getting us hacked, when they quickly released 1.3.22 to fix how they broke half of WordPress, I let it install that version.

Overnight, they’ve since updated to 1.3.23 to fix how they send things back to Cloudflare to pre-detect spam. So they made a giant mess and they’re clawing back at it right now.

There’s a problem that several people are reporting presently, that they can’t post comments while not logged in by submitting their email addresses — because the email address never validates. Clearly this is because the Cloudflare plugin is trying to sanitize that variable as well, incorrectly. Other blogs are also having this issue, as seen here: ERROR: The email address isn’t correct. (4 posts) and here: ERROR: The email address isn’t correct. (3 posts). This problem isn’t just impacting The Orbit, but any WordPress site that uses Cloudflare.

But because of the terrible nature of what they’re fixing here, we kind of have to ride out this storm. I could try to implement my own bugfix for this, e.g. by removing email address validation, but that would have other negative impacts on the rest of the site.

For now, please log in to make comments. Sorry for the inconvenience. Hopefully they’ll fix this issue too, as soon as possible.

This is a disaster and it was entirely avoidable through proper QA of the plugin before it being released. The rapid fire nature of the plugin updates speaks to a sort of panic to address the initial vulnerability, which is laudable, but a lack of foresight as to what kind of impact specific changes might make to the rest of the service. Those of us who rely on the plugin should not be stuck choosing between being hacked, being entirely unprotected against DDoS and spam, or having people be able to comment.

UPDATE: they released an update which properly namespaces their variables and only sanitizes those variables, so things should finally be under control. This is why you don’t release plugin updates into production without testing.

Try again to comment, please, folks.