If you don't want to see a specific recent contact when you double tap the home button, you need to remove the call log from 'recents' in the phone app. So, open the phone app, click on 'recents' and left-swipe and 'delete' any recent calls from that specific contact.