nginx

Manually renewing SSL certificates with Certbot

by
A screenshot of Putty connecting over ssh to a server running certbot, where the command has been issued to manually renew an SSL certificate. The domain has been pixelated.

Back in February, I started using nginx Proxy Manager to manage external access to the various web services that I host on my Raspberry Pi – namely, Home Assistant, calibre-web and Nextcloud. Nginx Proxy Manager (NgPM) includes Certbot, which is an automated tool for managing SSL certificates from Let’s Encrypt, and it should automatically renew certificates every three months so that there’s always a valid certificate in use.

In practice, this doesn’t work on my NgPM install. I understand it’s a bug in an older version that has been fixed, but as I run NgPM as a Home Assistant addon, that bug fix hasn’t made its way downstream. Attempts to renew the SSL certificates through the NgPM web interface fail with unhelpful errors.

Hopefully, the Home Assistant addon package will get updated soon, and this won’t be a problem anymore. But in the meantime, this is the workaround that I’m using – manually interacting with Certbot on the command line to generate a certificate. This can then be imported into NgPM manually.

Step 0: access Certbot through Docker

If you have access to Certbot directly, you can skip this step.I don’t, and Certbot is no longer supported on Windows, so I’m using the version of Certbot that comes with NgPM.

As this runs in Docker, we need to open a shell session inside the Docker image, using docker exec -it addon_a0d7b954_nginxproxymanager sh. I had to run this as root on my system using sudo.

Step 1: request the certificate

Now we can interact with Certbot itself. Here’s the command to type:

certbot certonly --manual --preferred-challenges dns - d example.com

Let’s break this down:

  • certonly specifies that we just want the certificate – we don’t want Certbot to install this for us.
  • --manual tells Certbot that we want to manually authenticate the domain.
  • --preferred-challenges dns means that we want to authenticate using DNS, rather than HTTP – this is tricky to do when you’re using a reverse proxy
  • -d example.com is the domain that we want the SSL certificate for.

Step 2: add a TXT record to authenticate

If you use something like Google or Cloudflare for DNS, then you may be able to use a plugin to automate this step. I don’t, so here we create a TXT record on our DNS provider’s dashboard to authenticate the certificate. This will be something like _acme-challenge.example.com and will include a text string that Certbot gives you.

Once you’ve created the TXT record, my suggestion is to set a timer for 2-3 minutes, before pressing Enter to continue. DNS records can take anything from a matter of seconds to a few minutes to propagate, and if you try to continue too soon, the authentication will fail and you’ll need to go back to step 1. Trust me on this.

Step 3: download the certificate files

If the authentication is successful, then Certbot will have created two files for you. For me, these were something like:

/etc/letsencrypt/example.com/fullchain.pem
/etc/letsencrypt/example.com/privkey.pem

As I was running Certbot from within Docker, the easiest way I found to save these was to type cat /etc/letsencrypt/example.com/fullchain.pem (and for privkey.pem) and then copy and paste the output into a file locally.

Step 4: add to Nginx Proxy Manager

If you’re using Nginx Proxy Manager and want to be able to use your new SSL certificate, then open the SSL Certificates tab at the top, click ‘Add SSL Certificate’, and then ‘Custom’. Don’t choose the Let’s Encrypt option; although these certificates were issued by Let’s Encrypt, you want to import them manually.

Give it a name – I usually put the name of the service and the month (e.g. Nextcloud Sept 2024). Upload the privkey.pem file as the Certificate Key, and fullchain.pem as the Certificate. Click Save.

Now, go to the Proxy Hosts tab, and choose the host that matches the SSL certificate that you’ve uploaded. Click on the three dots on the right hand side, and choose Edit. On the SSL tab, select the certificate that you’ve uploaded. And that should be it – try navigating to your domain to see if it’s working and check that the new certificate is in use.

No auto-renewals

It’s worth baring in mind that manually-issued Let’s Encrypt certificates won’t normally auto-renew. You apparently can use validation hooks to enable auto-renew, but this goes beyond my expertise.

I’m hoping that the package maintainer for the Nginx Proxy Manager addon for Home Assistant will issue a new release soon, which will enable me to auto-renew my certificates in future. If not, then I have my own guide to follow to manually renew.

Nginx Proxy Manager

by
A screenshot of nginx proxy manager

I’ve recently started using Nginx Proxy Manager to act as a reverse proxy for the various web services that I have running on my Raspberry Pi. It’s a frontend to the nginx web server and makes setting up reverse proxies and SSL access really easy.

You may remember that I’ve used SWAG for this before, to enable HTTPS access to my Home Assistant installation. SWAG is find if you’re using Home Assistant Core or Container, but having switched to Home Assistant Supervised recently, I needed a new solution. Like SWAG, Nginx Proxy Manager can be run in Docker, but it’s also available as a Home Assistant addon.

The main benefit of Nginx Proxy Manager over SWAG is that it has a web-based UI, shown in the screenshot above. This makes setting up new proxy hosts really easy, as it has a nice and simple interface. Whilst SWAG includes pre-built configuration files for many services, there’s no interface available other than editing text files.

Nginx Proxy Manager will also manage SSL certificates. You can either import ones that you purchase yourself, or it will manage the process of acquiring and renewing Let’s Encrypt certificates.

As well as managing Home Assistant, I have Nginx Proxy Manager looking after Calibre-web and Nextcloud.

A brief explanation of reverse proxies

Apparently, you’re supposed to ensure that blog posts are at least 300 words nowadays, otherwise Google ignores it. So, here’s an explanation of why you should set up a reverse proxy server if you’re hosting services like Home Assistant at home:

  1. No port numbers. When you set up something like Home Assistant, you’ll end up with an address like http://192.168.0.1:8123. A reverse proxy will allow web browsers to connect on the standard web ports (80 for HTTP, 443 for HTTPS), which looks nicer and is more predictable. It also means you don’t have to forward lots of arbitrary ports on your router.
  2. SSL certificates. If you’re running a device on your home network, then ideally you only want to allow secure connections to reduce the risk of your personal data being intercepted. Self-signed SSL certificates are not ideal, as most web browsers issue dire warnings for web sites that use them. As not all web applications support SSL certificates natively, a reverse proxy can handle this for you.
  3. Web application firewall. By making all requests go via a proxy, the proxy server can filter out malicious traffic. Nginx Proxy Manager includes a ‘block common exploits’ mode, and you can also filter IP addresses. For example, you may wish to only allow access to certain IP addresses.

Installing in Home Assistant

If you’re running Home Assistant Supervised or Operating System, then you’ll need to install the Nginx Proxy Manager addon. It’s available from the Community Addons repository, which should already be available to you – you won’t need to add it separately. It’s not to be confused with the official ‘NGINX Home Assistant SSL proxy’ addon; this doesn’t include an interface and only enables a proxy for Home Assistant, and not for any other services. Indeed, if you’re already using this official addon, you’ll need to stop it from running first, as otherwise you’ll have a port conflict.

Once set up, you can access the web interface at http://[your IP]:81 . I suppose I could probably set up a reverse proxy host to get rid of the port number, but I don’t see a good reason to enable remote access to it.

One final thing to add is that the user guide for Nginx Proxy Manager isn’t great. It covers setup, but there’s very little help for configuring proxy hosts. The web interface is pretty straightforward so arguably detailed instructions aren’t necessary, but a little more help would be good.

Home Assistant with HTTPS and HomeKit

by
A screenshot of Home Assistant running in a web browser with HTTPS enabled and no certificate errors

Welcome to the latest chapter of getting Home Assistant working on a Raspberry Pi using Docker. Last time, I’d managed to get it working in Docker, but only over a regular HTTP connection and without HomeKit. The good news is that I’ve solved both of these problems.

Using SWAG to enable HTTPS

Firstly, I recommend reading this paragraph whilst listening to ‘Swagger Jagger’ by Cher Lloyd.

I’ve tried lots of different ways to get Home Assistant working over SSL/TLS. There’s a good reason why this is one of the key selling points of Home Assistant Cloud, as it can be difficult. Thankfully, there’s a Docker image called SWAG (Secure Web Application Gateway) that handles much of the legwork. Once you’ve installed SWAG, follow this guide, and you should find that you can access your Home Assistant setup at https://homeassistant.[yourusername].duckdns.org/ . No need to specify a port, or accept any certificate warnings.

Inside SWAG, there’s a DNS client, which will automatically renew the SSL certificates every 90 days for you, using ZeroSSL or Let’s Encrypt. There’s also nginx, which is used to set up a reverse proxy, and support for dynamic DNS services like DuckDNS.

SWAG has sample configurations for lots of different services, including calibre-web, so I have SSL access to my calibre-web image too. My only issues with it so far were last week when DuckDNS went down on Sunday morning. Most services, like Home Assistant, need to be mounted as subdomains (as above), but others (like calibre-web) can be mounted as subfolders, e.g. https://[yourusername].duckdns.org/calibre-web. This reduces the number of subdomains that you need SSL certificates for; ZeroSSL only offers 3 subdomains for a free account so it’s worth considering subfolders if you want to add more services.

If you have your own domain, then you can also add a CNAME to it to point it at your DuckDNS account, should you wish to use that rather than a [something].duckdns.org address.

Getting Apple HomeKit working

Carrying on the musical theme, here’s ‘Carry Me Home’ by Gloworm, a 90s dance classic which has only recently become available on digital platforms again.

After getting my swagger jagger on and getting HTTPS working, the final issue I’ve been having with Home Assistant is the HomeKit bridge. Adding Home Assistant devices to Apple’s Home app is something that normally works out of the box if you install Home Assistant OS, but takes more work if you use Docker.

The instructions which helped me where these on the Home Assistant forums. You’re going to need to install another Docker image containing avahi; there are several but this one worked for me. It’s bang up to date, unlike the most common Docker image which is, um, 8 years out of date and also only works on x86 machines. Which isn’t much help for my arm64-based Raspberry Pi 4.

Once you’ve installed avahi, added the relevant lines to configuration.yaml in Home Assistant and restarted it, HomeKit should work. To get started, add the HomeKit integration to Home Assistant – you may want to specify which devices will show if you don’t want all of them. Then, use your iPhone or iPad to scan the QR code in your Home Assistant notification panel, and add the bridge. If all goes well, it should immediately tell you that it’s an unsigned device, but will then let you set up each device in turn.

If it just sits there for several minutes and then gives up, you’ll need to do some more digging. Don’t worry, this happened to me too. I suggest downloading the Discovery app, which shows all of the mDNS devices broadcasting on your network. If you can’t see ‘_hap._tcp’ in the list, then there’s a problem. In my case, this turned out to be because my Raspberry Pi wasn’t connected to the same wifi network. It’s plugged in to my ADSL router with a network cable, but we use Google Wifi which results in a ‘double NAT’ situation. Connecting the Raspberry Pi to both wired and wireless connections seemed to fix the issue.

Indeed, as a side effect Home Assistant managed to autodiscover some additional devices on my network, which was nice.

Home Assistant Core in Docker? Done it, mate

All in all, I’ve successfully managed to get Home Assistant to where I want it to be – self-updating in Docker, secure remote access, and a HomeKit bridge so that I can ask Siri to manage my devices. I’m looking forward to being able to turn my heating on whilst driving, for example.

It’s been a challenge, requiring a lot of skimming through the Home Assistant forums and various StackExchange discussions. Ideally, I would have a spare computer to run Home Assistant OS, which would have taken some of the leg work out of this, but I’m happy with the setup. Finding SWAG and getting it to work was a moment of joy, after all the setbacks I’d had before.