The problem with your argument is that these infringements are never temporary. Once you give that in, they won't come back.You never had and never will have unlimited rights. Nor should you.
You don’t get to speed in your car.
You don’t get to shoot people.
You don’t get to commit ritual sacrifice.
You don’t get to yell fire in a crowded theater.
There is a long list of limitations on YOUR rights to protect MY rights and the rights of others.
As for your claim that “rights” when taken away never come back, history has proven you wrong on that too. Case in point, WW2. Numerous limitations were placed on the public due to the situation. Rationing of things like gasoline, rubber, nylon, and various foods. Price and wage controls. Limitations on certain leisure activities such as driving, and of course the draft (not to mention the terrible internment of Asian Americans). Outside of the US some areas, particularly those on the front had even greater restrictions, such as no lights after certain hours. And when the threat of war was over, these restrictions were removed.
If your “theory” was right all those things (and more) would still be in place today. Temporary impositions on normal freedoms happen during a crisis, whether it be short term such as a flood or hurricane, or longer, such as a war. Your paranoid theory is therefor proven wrong.
The examples of WW2 are weak. We live in a much more surveillance society than back then.
Also your examples of unlimited rights are based on the killing/safety of people. There is not denying that, it's common sense. The Covid issue is different because we are still learning the effects and causes of the virus. The CDC keeps changing their guidance in an almost monthly basis.