#011 - Deploying Secure HTTP Response Headers
I have a basic understanding of HTTP response headers, but when I discovered that Google hid my website, I decided to investigate and climb the rabbit hole.
I am convinced my spirit animal is a cat, because I seem to grow far too curious when I tinker with technology.
Two weekends ago, my Saturday was consumed because I was frustrated with how Google placed this blog, especially Googling the same term as the TLD name. At the time of writing, thejoyofprogramming.com was hovering around on page 13. Other times, the website would randomly vanish from the search results, which I never experienced before. Meanwhile, other search engines including Bing, Yahoo, Yandex, AOL, DuckDuckGo would plug the website in the first page of results.
I know newly 'branded' websites can take time to be crawled and depends on SEO (Search Engine Optimisation) and marketing, then more folk will naturally discover the website organically, which I appreciate. However, I was compelled to investigate when I noticed that Google would randomly hide my website from the Google search.
You would think there was foul play, or perhaps I violated the terms and conditions. Neither of which makes any sense. I know for sure that my content is clean - so there is no concern there. I also deliberately chose Ghost CMS over WordPress in the hope to avoid bloaty code and boost the SEO using a cleaner blogging platform.
I knew I had a wee bit of an issue with the SSL certification, as I am using Cloudflare to process the DNS and certification over the use of Let's Encrypt. I have used Cloudflare in another lifetime and I have never had any problems before. Again, there was nothing obvious with my settings as I read through several blogs of other people's experience with Cloudflare and DigitalOcean droplets.
Upon further Googling, I stumbled upon a recently published article that utilised DigitalOcean and Cloudflare - but with advanced settings to the SSL certification on Cloudflare. I decided to follow suit. I realised that after following through with the tutorial that I could no longer schedule posts on Ghost CMS, which is a key feature for me.
Who would have thought upgrading Ghost CMS from 2.9 to 2.16 is even a thing? So, of course the post scheduler failed at this point.
I know WordPress delivers on the same post scheduler as a key feature and I'm aware that WordPress is much more user-friendly when upgrading, but I want to survive without WordPress - at least, for a wee while longer.
I slept on the issue overnight and decided to tackle the whole problem in one go from Saturday morning.
I decided to fire up a new droplet on DigitalOcean and ported my floating IP address, which I consider an essential feature on DigitalOcean. I also exported my content from the old Ghost CMS droplet to the new droplet.
I was hoping Ghost CMS would be smooth like WordPress when it came to importing content. Ouch! No chance. I had to reconfigure the author, tags and posts and re-upload the hero images used for each published post. Even the saved JSON export wasn't compatible with the (seemingly) latest copy of Ghost CMS. As a result, I had to manually copy the latest blog post.
In addition, I deactivated the Google Accelerated Mobile Pages (AMP) module within Ghost CMS, as I read that Google is strict in seeing AMP-embedded code within a given theme, even though it can appear as an AMP page without customising a theme directly. I also toggled off the Google AMP setting and deselected the minify of the HTML, CSS, JavaScript options within Cloudflare.
The other gripe with Ghost CMS is when you make any significant changes to the theme, you need to activate another theme and then re-activate your favoured theme to see the changes. Any live changes made are not shown using the simple refresh command - whether it's the F5 key or CTRL+R on the keyboard. Curiously, WordPress can handle dynamic changes - be it live using the GUI or the backend code.
With Ghost CMS finally working again - including the post scheduler, I dived into the world of HTTP security headers. I freaked out when my blog was vulnerable to attack, as it was missing settings to the following HTTP security headers: Content-Security-Policy, Referrer-Policy, X-XSS-Protection, and Feature-Policy.
I am certain the blog scored a pathetic D via: https://securityheaders.com/
After learning to deploy one-line add_header settings via /etc/nginx/conf.d on Nginx, I eventually secured an A+ on the Security Headers website.
I am hoping the redeployed website and HTTP security headers and updated SSL certifications will improve the Google search of this website. I don't have use any affiliation or marketing promotion, because it's a personal blog. I also tinkered with the HTML meta tag and added a fuller website description that I used in my Hello World page.
At the original time of writing, the website was appearing on page 10 of Google. A week later and the website has jumped to page 7, which I suspect was related to my recent post regarding The Division 2 and the Tobii Eye Tracker. So something is working.
Either way, time will tell, but I'm glad I pushed myself to consume much of my Saturday, as I wouldn't have come to learn of the importance of hardening HTTP response headers. I'm surprised a single line of code can help to shield websites from man-in-the-middle attacks, but you can add such security measure if you manage your own web server - or for me, it's a VPS (virtual private server).
It may explain why shared hosted websites are susceptible to exploits and hacks, due to the web host's negligence to deploy such basic functionality. At the end of the day, your average consumer won't care to know the technical details - other than, can it host a WordPress blog.
P.S. For a deeper understanding of HTTP response headers, there is a good overview on the Mozilla Developer Network at: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
~Richard