Dynamic range has nothing to do with sampling rates: it is a property of digital word length / precision - ie 24 bit digital has a higher dynamic range than 16 bit digital because it encodes a much larger range of possible values. (I studied audio as part of my engineering degree.)
Regardless of the noise floor, tape is inherently less “lossy” because it has - at least theoretically - an infinite sampling rate and infinite precision.
I agree, digital potentially provides a higher dynamic range, given sufficient bits. That said, tape has the advantage of saturating much more gracefully than digital (ie is more forgiving of incorrectly set levels for sounds that are louder than unexpected). But we are getting off track 😉
At no point did I advocate tape over digital formats (though it should be noted that the quality of tape is very much dependent on tape width and speed and it has its advocates)… I never even mentioned tape. (Tape is an analogue recording format but it is not what I mean by analogue.)
When I say that, by your definition, the only lossless format is analogue, I do so because the moment a signal is digitised, information is lost because of the inherent nature of digitisation. An analogue wave has an infinite number of samples at infinite resolution. A digital signal has a finite number of samples at a finite resolution. Therefore, by your terms, all digital is inherently lossy.
All that said - conventionally - lossy and lossless refer to the type of file compression algorithm rather than the sampling rate and precision.