First requests to a site no longer wait several seconds
A site could take 3 to 13 seconds to answer the first request after a gateway's 30-second domain cache expired, because the control plane was starved of processor time. It now has dedicated capacity and does far less routine work.
A gateway remembers which site a domain belongs to for 30 seconds, then asks our control plane again. That lookup had become slow, so the first request to a site after the 30 seconds ran out could wait 3 to 13 seconds for an answer. Requests in between were not affected.
The control plane was being given only a small share of the processor it runs on, and was spending much of that on work nobody needed. Both are changed.
What changed
- Dedicated capacity. The control plane no longer shares its resources.
- A full storage listing ran on every node check-in. It now runs once a minute, and every check-in in between uses that result.
- A record was written on every check-in that nothing ever read. It’s gone.
- Calls to a cache that was refusing them were retried continuously. They are now paused for a minute after a failure and tried again once, and a reconnect loop to the same cache was removed.
The trade-off
A deploy deleted from storage is cleared from nodes’ caches up to a minute later than before. Visitors are not affected, because the gateway stops serving a deploy as soon as its domain points elsewhere.
What we measured, and what we didn’t
The control plane’s own load was measured before and after these changes. Response times on customer sites were not measured again as part of them, so if the first request to your site is still slow, tell us.
What you need to do
Nothing.