When you install an app from the App Store the installation is tied to the Apple ID that is currently logged in to App Store on the device.
If you log out and log in with a different Apple ID, as you've done, all apps installed on the device continue to be tied to the Apple ID they were installed with. Updating those apps requires the password for the Apple ID they were installed with, not for the Apple ID currently logged in on the device.
It sounds to me like you're entering the password for the second Apple ID, when you should be entering the password for the first Apple ID (the one the apps were installed with). If this is the case, you have two options:
1) Delete all apps installed with the previous Apple ID, and reinstall them from the App Store (if they're paid apps, you need to pay for them again unless both your first and second Apple ID belong to the same family and the apps support Family Sharing)
2) If you don't want to delete the apps, you need to continue to enter the password for the first Apple ID (the one they were installed with) whenever you want to update them. Also, to my knowledge the apps will not update automatically while a different Apple ID is logged in to the device - you'll need to manually check for updates