A page that breaks every time gets fixed by lunch. A page that breaks one time in three gets told it is fine.
This week a collection page for a homeware store I work with did something odd. It laid the products out three across, then a beat later shuffled the whole grid into four across. A shop assistant rearranging the window while you are standing at it, nose to the glass. Google scores that kind of jump, and anything over 0.1 counts as a bad experience. This one scored 0.26.
The first check loaded the page once. It came back clean at 0.012 and got filed as not reproducible.
That was the mistake.
Here is what was actually happening. One styling rule said three across and arrived at the top of the page. A second rule said four across, and it had been typed into the theme editor’s custom CSS box. Shopify prints that box at the very bottom of the page. On this one, 99 per cent of the way down, after roughly 900,000 characters of everything else. The browser dresses the window, strolls to the bottom, finds the new instruction and moves every product. Whether a shopper sees it depends on whether the browser painted before it got there. A coin flip with the furniture.
Load it nine times and three of them jumped. Load it once and you get to pick your story.
The fix was one line, the same rule copied into the stylesheet that already loads at the top. Live afterwards, eight loads, zero jumps. Someone had also suggested tidying away some “duplicate” CSS elsewhere on the site. That duplicate does exactly this job, so it stays.
A bug that turns up one time in three does not need a test. It needs a count.
One clean run is not a clean page. It is a coin that happened to land heads.