I was thinking about getting one for when I eventually go out on deployments (I'm in the Navy). That way, it's less likely to get stolen and I won't have to worry about it as much because it can take the abuse, my D5100 can't.
I'm not sold on the "less likely to get stolen" thing. Most non-photographers don't know a D1 from a D4 from a D3000. To them, the bigger the camera (or the longer the lens extension) the more expensive, and better it is. If anything an old D2H is
more likely to get stolen because it "looks" more like a pro camera. Thieves aren't going to rifle through your stuff and think "well, that's only a D2H, nah I'll pass on that" and walk away.
As far as if it will actually be better, it really depends on your intended usage. And don't discount the performance of your D5100 because it's supposed to be "entry level". The focusing units on entry-level to midrange bodies is usually the high-end focusing module from the pro bodies 2-3 generations back. Specifically in this case, the D2H uses the Multi Cam 2000 focusing module, and the D5100 uses the Multi Cam 1000 module, an only slightly-worse derivative of the 2000 which was first introduced on the D200. Frankly, AF performance between a D5100 and D2H are going to be very similar. It is by no means unreasonable to assume the Multi Cam 1000 on the D5100 is easily as good or even better than the older generation Multi Cam 1300 found on the D1 series cameras.
As far as ergonomics goes, go back and actually look at the D1 series cameras. I think you will be surprised at how few buttons they have, and how far ergonomics have come on the D5100. Not to mention you will have lost Live View functionality, and also taking a MAJOR step backwards in terms of the rear LCD.
IMHO you are giving up way too much in the imaging chain to compensate for "pro body ergonomics", the merits of which are also debatable. Unless you have a very specific need for high-fps shots (and if you think you do, don't forget to read this recent thread very carefully:
https://forums.macrumors.com/threads/1366563/) and only intend to publish on the web, then I don't think it's worth it to get a D2H. Despite what people tell you, 4.1MP is sorely lacking for modern prints unless you keep the output size really small. 4.1MP is a 4x6" print at 360ppi. Barely above 5x7 at 300ppi. At 200ppi, where you are really getting to the marginal definition of "tack sharp" you're still only at around 8x12.
You're also losing significant high ISO capability, and so unless you have an arsenal of only the fastest (as in f1.4) lenses to back it up, all you're going to get from a D2H are 7 blurry, grainy, and unusable shots per second in marginal light. Whereas with the D5100, you're limited to fewer fps but can at least get much more usable pictures.
As far as build quality goes, yes a D2H is built better, but unless you find one being sold by a rich doctor or lawyer who's stepping up to the latest and greatest, it has also experienced years worth of additional wear and tear. You'd be surprised at how tough/resilient the D5100 is. Additionally, with its built-in dust removal system, the D5100 would arguably fare even better than a D2H in dusty conditions.
Newer derivatives like the D2x and D2xs still do hold their own, as their imaging capabilities are still very capable- but they are also quite a bit more expensive. I'll agree with Vantage Point that a D300 is in a great sweet spot right now as far as getting a dated body on the cheap that still has a very modern, capable imaging chain, and has "pro-grade" build and features.
IMHO, you're better off saving your money to put towards a more modern, more capable body. Sounds like you are just fantasizing about using a "pro" body with "pro" build and "pro" features, without really needing them, or without really realizing how "pro" your D5100 really can be when used properly.
Think about it this way: if you had both a D5100 and a D2H, and carried them both everywhere you went, which one would you want to be shooting with to produce optimum results? I know if it were me, I would prefer the D5100 every time, except if I absolutely
had to shoot at 7fps, or if I were absolutely
forced to shoot in a driving rainstorm (even then, I could get external rain protection for my D5100 and probably get better images). Every other instance, I want the D5100- so times I would actually prefer and benefit from the really old pro body are exceedingly small.