Blend Developer Guide

How do I convert a site to static files?

  • Page Owner: Not Set
  • Last Reviewed: 2021-02-11

I need to generate a static version of a CMS-driven (or otherwise dynamic) website. How can I do this?

Answer

I'd recommend the website-scraper NPM module and a quick Node app.

For example, the script used to generate static site for Mercury looks something like this:

    const scrape = require('website-scraper');
    
    scrape({
        urls: ['http://old.mrcywebdev.com/'],
        directory: './output',
        recursive: true,
        maxRecursiveDepth: 50,
        urlFilter: function (url) {
            return url.indexOf('http://old.mrcywebdev.com/') === 0;
        },
        request: {
            headers: {
                // You may need to log into the site and grab your cookie value to be logged in.
                'Cookie': '<SNIP>'
            }
        }
    });

(Note: the full script also uses a plugin to customize the output a bit. See the repo for full details.)