You used to have a link to a folder/file up there. You must have deleted the folder/file, the link remains but OSX can't find the file/folder so it replaces the icon with a ?.
Like if you delete an application without removing it from the dock you will get the same ? when you try and launch the application.