I’m not sure where you found that confirmation. Nginx would downright refuse to start if the certificates aren’t where they’re supposed to be. However, HTTP is still working, so that’s not the case.

I would suggest taking a look at your nginx error log.

Again, changing your client invocation or Let’s Encrypt’s configuration files won’t change anything. Right now, your web server is not responding to HTTPS requests. You’re redirecting to an URL that doesn’t work. That is what you need to fix.

You confirmed it. But NGINX didn’t die, it only refuses to open the sites that redirect all request to port 443 and that LE failed to update; another site (www.stigharder.com) on the same server that listens only to port 80 is still up. That site failed also, but as it needs to be live, we changed the config not to use TLS until this issue has been resolved.

That’s correct and this is the reason I opened this thread. The site was up until LE tried (but failed) to update the certificate. According to @pfg , it’s because NGINX refuses to open a site “if the certificates aren’t where they’re supposed to be.” We didn’t change anything; all was working perfectly until the failed update by LE. The minute cron started the updates in sequence, all the sites that had LE certificates in place promptly died one after the other.

Hey @pfg – we just ran /opt/letsencrypt/letsencrypt-auto certonly --renew-by-default --config /opt/letsencrypt/webroot.moonloupe.com.ini again and the error log reports:

2016/02/04 17:44:04 [error] 484#0: *5215 no "ssl_certificate" is defined in server listening on SSL port while SSL handshaking, client: 66.133.109.36, server: 0.0.0.0:443

This server block doesn’t have a ssl_certificate directive (or any of the other directives necessary for SSL to work):

server {
        listen 443;
        server_name example.com;
        return 301 https://www.example.com$request_uri;

Make sure you include all SSL directives in all server blocks that use SSL.

Alright, we added everything to the redirect block, but now NGINX failed to restart:

server {
        listen 80;
        server_name moonloupe.com;
        return 301 https://www.moonloupe.com$request_uri;
server {
        listen 80;
        server_name www.moonloupe.com;
        return 301 https://www.moonloupe.com$request_uri;
server {
        listen 443 ssl;
        server_name moonloupe.com;
        return 301 https://www.moonloupe.com$request_uri;
        ssl on;
        ssl_certificate /etc/letsencrypt/live/moonloupe.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/moonloupe.com/privkey.pem;
        ssl_stapling on;
        ssl_stapling_verify on;
        ssl_prefer_server_ciphers on;
        ssl_session_cache shared:SSL:10m;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SH$
        add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
        ssl_dhparam /etc/ssl/private/dhparams.pem;
server {
	listen 443 ssl;
	listen [::]:443 ssl;
	server_name www.moonloupe.com;
	root /var/www/moonloupe.com/html;
	index index.php index.html index.htm;
	location / {
		try_files $uri $uri/ =404;
	location ~ \.php$ {
		include snippets/fastcgi-php.conf;
		fastcgi_pass unix:/var/run/php5-fpm.sock;
	location '/.well-known/acme-challenge' {
		root /var/www/moonloupe.com/html;
		try_files $uri /$1;
	ssl on;
	ssl_certificate /etc/letsencrypt/live/moonloupe.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/moonloupe.com/privkey.pem;
	ssl_stapling on;
	ssl_stapling_verify on;
	ssl_prefer_server_ciphers on;
	ssl_session_cache shared:SSL:10m;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA;
	add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
	ssl_dhparam /etc/ssl/private/dhparams.pem;
$ sudo systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled)
   Active: failed (Result: exit-code) since Thu 2016-02-04 18:49:19 EST; 20s ago
  Process: 28748 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
  Process: 28751 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=1/FAILURE)
 Main PID: 480 (code=exited, status=0/SUCCESS)
Feb 04 18:49:19 newyork nginx[28751]: nginx: [emerg] invalid number of arguments in "ssl_ciphers" directive in /etc/nginx/sites-enabled/moonloupe.com:27
Feb 04 18:49:19 newyork nginx[28751]: nginx: configuration file /etc/nginx/nginx.conf test failed
Feb 04 18:49:19 newyork systemd[1]: nginx.service: control process exited, code=exited status=1
Feb 04 18:49:19 newyork systemd[1]: Failed to start A high performance web server and a reverse proxy server.
Feb 04 18:49:19 newyork systemd[1]: Unit nginx.service entered failed state.
harder:

ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SH$

this line is cut off (see trailing $).

Hey @pfg! Wow, that did indeed bring moonloupe.com back up and we were also able to update the certificate with /opt/letsencrypt/letsencrypt-auto certonly --renew-by-default --config /opt/letsencrypt/webroot.moonloupe.com.ini. The odd thing is that the new certificate doesn’t seem to connect with the site. The new certificate we just made expires on May 4th (see the output below), not May 2nd, which was the expiration date of the one we originally made two days ago.

$ /opt/letsencrypt/letsencrypt-auto certonly --renew-by-default --config /opt/letsencrypt/webroot.moonloupe.com.ini
    Updating letsencrypt and virtual environment dependencies......
    Requesting root privileges to run with virtualenv: sudo /home/harder/.local/share/letsencrypt/bin/letsencrypt certonly --renew-by-default --config /opt/letsencrypt/webroot.moonloupe.com.ini
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/moonloupe.com/fullchain.pem. Your cert will
   expire on 2016-05-04. To obtain a new version of the certificate in
   the future, simply run Let's Encrypt again.
 - If you like Let's Encrypt, please consider supporting our work by:
   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le
listen 80; server_name www.moonloupe.com; return 301 https://www.moonloupe.com$request_uri; server { listen 443 ssl; server_name moonloupe.com; return 301 https://www.moonloupe.com$request_uri; ssl on; ssl_certificate /etc/letsencrypt/live/moonloupe.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/moonloupe.com/privkey.pem; ssl_stapling on; ssl_stapling_verify on; ssl_prefer_server_ciphers on; ssl_session_cache shared:SSL:10m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA; add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"; ssl_dhparam /etc/ssl/private/dhparams.pem; server { listen 443 ssl; listen [::]:443 ssl; server_name www.moonloupe.com; root /var/www/moonloupe.com/html; index index.php index.html index.htm; location / { try_files $uri $uri/ =404; location ~ \.php$ { include snippets/fastcgi-php.conf; fastcgi_pass unix:/var/run/php5-fpm.sock; location '/.well-known/acme-challenge' { root /var/www/moonloupe.com/html; try_files $uri /$1; ssl on; ssl_certificate /etc/letsencrypt/live/moonloupe.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/moonloupe.com/privkey.pem; ssl_stapling on; ssl_stapling_verify on; ssl_prefer_server_ciphers on; ssl_session_cache shared:SSL:10m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA; add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"; ssl_dhparam /etc/ssl/private/dhparams.pem;

You’ll need to reload your web server, usually with something like service nginx reload (at least on Ubuntu).

Nginx will drain all existing open connections and then reload the configuration and certificates. This works without downtime.

Oh, yeah – that did the trick! It’s funny – we now added all the TLS information to the redirect block, but the “.well-known/acme-challenge” part is included only in the final block. This all seems very eccentric to us, something unique to LE (not that we don’t love eccentric – we do!); the other sites we run, including fashion.net, do not have anything TLS related in the 443 redirect. In any case, as this works now, well just update all the configuration files and all will be good again. Thank you, @pfg – absolutely amazing!