I understand that. In general, I don't have any preferred contacts that I want to allow when Do Not Disturb is enabled. That's my point, therefore, whoever randomly calls doesn't know I have Do Not Disturb enabled.
Record your voicemail message to say you have DND on. Save you having to explain all the time.