Can a browser detect if a screenshot was taken?

Not really, but there might be some DRM involved to prevent the screenshots from being usable.

The contents on screen is actually not rendered directly. Every window has a rendering context that is than assembled to produce the actual screen contents.

If it were any different, moving a window would require every app to provide its content again.

Taking a screenshot actually means to iterate over all visible windows and ask them to render in a separate context. This context is than saved as the image.

The desktop background for example belongs to the finder.

So in theory, an app can check what context is used to render itself into.
You can see that on Apple DVD player if connect an HDMI adapter.

The window of the player, and only that window, will produce a checker pattern to prevent showing copyrighted material on an external screen.

Most apps use standard API that display GUI elements and are not programmed to do this kind of check. But it is possible.

Can your browser do that? Probably not.
Can a browser plugin implement this check? Maybe.

If your test uses a custom plugin to display content I would assume that it's possible to detect if you take a screenshot. But they probably have no idea how to implement such a check.

A click outside the window will tell the plugin that its window is no longer the active window. That's easy to react to.