Display flickers after unplugging external monitor

I've noticed that over the past few weeks, whenever I unplug my external display from my Mac, the display flickers for a few seconds. When I try to change the brightness of the display when it's flickering, it won't let me (just goes back to the original).

The flicker only happens when unplugging the external display.

Any ideas?