I think you have dragged a file onto your Finder's bar. To remove the icon:
cmd-click on an empty space on the Finder bar, select edit tools (or what it's called in English, I have a Norwegian version of OS X running here), click and drag the icon off the bar.
You dragged a file there, then deleted it. Now it can't be found, so you get the question mark. I think you can just command-click on it directly and drag it off.