Auto generate cached pages

1

Hello,

I have a big website in development (~1000 pages). When I first access a page, I have a pretty long loading time (4-5s). Susbesquent acces are fast as expected (~400ms). From what I understand of grav, it is because it needs to cache the page. And I can see this info on the first access:

Flex: Caching user-groups index of 4 objects

And in this case, most of the duration is taken by Init frontend routes.

It seems the cache of the pages are not generated automatically. Is there a way to auto generate cache for all pages of the website?

2

I have a similar large site and use a command from my development machine after major updates or cache clears. It uses wget, a http client written for Linux, to grab every page on the site:

wget --spider -o warm.prod.log -e robots=off -r -l 5 -p -S --header="X-Bypass-Cache: 1" -w 1 -U "Wget cachewarmer" https://yoursite.example.com

There are many options available in wget. Here are some used here. I must have copied this from somewhere else initially because some options are obscure in the docs:

  • -o gives me a log file
  • -e robots=off tells wget not to look at /robots.txt
  • -r -l 5 fetches resources/pages recursively but limits recursion depth to 5 levels. It’s a safety mechanism but you might want more depth.
  • -p makes sure everything needed to render a page is retrieved and therefore cached (images, stylesheets etc)
  • -S output server responses
  • --header="X-Bypass-Cache: 1" bypasses the server cache and I think you can just use --no-cache instead of this
  • -w 1 forces a one second wait between requests to lighten the load on your server. You can set this much higher if your site is busy.
  • -U .. allows you to set a name for the client (wget) User-Agent string of your HTTP requests, which will help you identify them in log files and metrics dashboards (so you can ignore them etc).

Another plan is to use a plugin. I have never used one. If you search plugins for “cache” you will see a few, but many are old. Warm Cache is probably a good option if you have a Premium account or want to pay for it. The free options are much older but may work.

It would be good if this capability was included in Grav’s Admin and command line tools along with clearing the cache. I don’t think that will happen because there is a premium plugin.

2 Likes
3

Thanks for your detailed answer. I’ll keep you informed of my results!

4

Take a look at the official Grav Plugins page:

There are PreCache and AdvancedPageCache plugins (for free), as well as Warm Cache plugin ($20).

Personally, I use the first one.

5

Hello,

Thanks a lot for your feedbacks. I installed Precache and it has a useful feature of logging what it is caching. It seems I have a problem with my cache : It is recomputed too often even if I don’t make changes in my pages : my cache is rerendered after a while when I go to the website, after more or less 1 hour.

In my configuration I have set the cache lifetime to 0 seconds (infinite) and also all different Flex caches to 604800 (1 week). Cache driver is apcu , and I tried redis, file, with the same problem

As my index is listing all the pages of the website (with title and a few custom fields), I am thinking this is perhaps the cause of the problem?

But the problem is important because after 1h I have a very long first build of my index. For example 56s (38s for Init frontend routes, and 16s for render).

Do you know what I can do to prevent this behavior? or what I am doing wrong?

1 Like