When you first turned File Vault on, it gave you the choice to use your iCloud password if you need to reset the password, or to not use iCloud, and FileVault would generate a password for the recovery key.
You do remember which choice you made, right?
Either way, you have primary access with your admin password.