W0lf is correct. However, it certainly doesn't seem like Apple is making it easy. I was able to dig through the folders, but had to make an alias for the encompassed folders each and every time.
If you try to navigate just by double clicking a folder (without creating an alias), you get a blank iCloud Drive view with nothing in it.
Hopefully somebody will find out an easier way to do this.