brainbaking/content/post/2021/12/migrating-from-mailchimp-to...

5.8 KiB

title date categories tags
Migrating from Mailchimp to Listmonk 2021-12-29T14:19:00+01:00
software
privacy
go
Listmonk
Mailchimp
self-hosted

In January of 2022, Mailchimp's General Data Protection Regulation (GDPR) is about to change. Why? Intuit acquired Mailchimp, requiring a slew of legal-related changes, such as terms of use, data processing options, etc. I don't do mailing lists myself, but my wife uses Mailchimp for her small start-up to send out monthly "moments of revelation" or lichtpuntjes (Dutch people: take a look and subscribe!).

Trained as a lawyer, she of course is very mindful of data regulation issues. Mailchimp's off-site---outside of the European Union---server location was always a bit of a pickle, and now with the acquisition, things got worse. According to her, they're not complaint with the latest GDPR rules and still hiding behind the now dead EU-US Privacy Shield. Long story short: we wanted to take control of her data: the subscribers.

An excessively long list of Mailchimp alternatives didn't make the choice any easier. Elimination of outside-of-Europe server locations did shorten it by quite a bit, but one of the other problems with Mailchimp-alike solutions is its needless complexity. My wife and me regularly got lost in the menu navigation and sheer volume of options, while all we wanted to do is compose a mail and send it out.

My growing interest in the Go programming language made me stumble upon Listmonk.app a few months ago. This was the perfect time to try it out. It's essentially a simple and self-contained self-hosted solution that uses a Postgres DB to store the sensitive data, meaning we can decide where to physically put it. The installation was dead-easy: ./listmonk --new-config and ./listmonk --install. Yay, we've got an instance up and running!

The only downside (and perhaps also upside) of simpler alternatives is the focused usability: Listmonk is nothing but a shell that manages your subscriptions and email campaigns. The actual sending of the mails, through SMTP or whatnot, is not done through the application, but requires the configuration of an external mail server. After failed attempts to try and get it to work with smtp.gmail.com, I was ready to de-install it. Until I thought "what am I doing?"---email is not secure, the headers are never encrypted, I'm still giving Google all contact information. Why would I do that?

The alternatives are MailGun, Coresender (which even has a nice Listmonk + Coresender install page), and the like, that are very expensive if your mailing list only contains 30 subscribers. Since I had a Postfix install up and running anyway, why not try to configure SMTP server localhost? First tests are cautiously optimistic. I am just waiting to hit the DMARC-failure-wall, but so far, we've tried sending it to a few different servers, without activating any spam filter. Great!

In case my future self is looking to reproduce the success, here are the steps I took to install and configure everything:

  1. Download, extract, and configure listmonk.
  2. Create a listmonk Postgres user, database, grant rights. (CREATE USER listmonk WITH PASSWORD 'jaddajadda';)
  3. Create a listmonk Linux account. Add hostname:port:database:username:password to 600-chmodded .pgpass for backups.
  4. Set SMTP server at localhost:25. Pray you've got things in order at the Postfix side.
  5. Configure lists and whatnot in Listmonk.
  6. Add a pg_dump command to crontab to timely auto-backup.
  7. Add a listmonk.service in /etc/systemd/system.
  8. Update all ansible scripts to automatically execute all previous steps!

My wife's site is a Wordpress install, meaning it uses PHP. I created a small server-side security check to avoid spammers from subscribing, and to anti-corrupt the Mailchimp interface. Thanks to that layer, it was easy to plug out and replace with a new one for Listmonk. It uses curl to call the API /Subscribers to subscribe new members to the respective mailing lists. Something like this:

// $tag is the human-readable mailing list name
public function subscribe($email, $tag) {
	if(!$this->validateMail($email)) {
		return $this->error('email');
	}
	if(!$this->validateTag($tag)) {
		return $this->error('tag');
	}
	
	$url = self::$server . "/api/subscribers";
	$payload = '{ "preconfirm_subscriptions": true, "name": "anonymous", "email": "' . $email . '", "status": "enabled", "lists": [' . self::$tags[$tag] . '] }';	
	$json = $this->curl($url, "POST", $payload);

	return $this->success();
}

Listmonk uses numbered IDs in the lists array, which kind of sucks, so associative array $tags "solves" this. Lastly, since we don't want to collect a name, but the API still requires one, I simply submit "anonymous".

So far, we're happy with the switch. A few things to take into account:

  • There's no Dutch translation available---yet. I'll try to do the translation in the coming days.
  • In Mailchimp, you could send a "welcome!" mail after each subscription. I can hack my way around this but haven't found the feature here.
  • The content template uses Go's templating engine. While I love this, it's a bit difficult for my wife to use, and there's no box to pick template keys from.

Hopefully the next campaign that launches will successfully land in every mailbox! At least now we fully control everything and can simplify our own privacy statement.