Have you check the CD's & DVD's preference panel in the System Preferences? There's a pull down menu for inserting DVD's. Just tell it to open DVD player when you insert a DVD. If that's okay, you might want to delete the DVD preferences and com.apple.DVDPlayer.plist
This should reset the DVD player and you should be able to set it up again the way to like it. Hope this helps.