If you want to turn on FV and it not take several hours, do so during a fresh drive install of the OS. If not, turn it on and be patient. I had no problems with it and was able to use my computer during the process, although doing that can make the process longer.
If I remember correctly, it took about 2 hours for me (as I turned it on after the install) and I only had 130 GB (give or take) on my 1TB SSD.
As a counterpoint, I'm booting from an external (USB3) Samsung 840 Pro. I formatted the drive as part of the High Sierra install process, but forgot to enable FileVault during the install. I enabled FileVault once the OS was loaded. We're on day two and it's just over half done. I'm giving up and going to reformat and reinstall. That kind of speed is... not acceptable.
(Edit: The machine is unusable while this is going on.)
Someone probably should have checked to ensure APFS I/O speed doesn't go to #@$% when enabling FileVault or using USB3. Or perhaps I've triggered some weird edge case. I will note that FileVault seems much, much slower with APFS even on a (different machine) maxed-out Mac Pro with internal SSD. I was under the impression that iOS stored everything encrypted, so you'd think all of this would've had a shakedown already when APFS was deployed there.