Let's be honest, football is the only sport that matters.

It goes back to when R.C. Slocum was fired. Was that ever a stupid move. Since Slocum was fired: 1 Top 25 final ranking (last year), Zero B12 championship game appearances,and Zero bowl wins. Surprisingly, they actually have a decent record vs. the Bovines during that span. Yes, I realize they were declining before that, but the program went off a cliff after he was fired.
Yep. Hiring the wrong coach can set you back for
years. That's what happened when RC was fired (I understood the thinking at the time but never agreed with it) and Franchione was hired. Since the 2003 season (Franchione's first) began, we've pretty much been terrible outside a couple of good years (2006, 2010) and a couple of mediocre ones (2004, 2007). Last year and the talent we have returning this year gives us hope that those days are gone.
Despite the fact that we've sucked for most of the last decade, we are historically a Top 20-25 program in terms of wins, conference championships, etc. Our history is most comparable in the SEC to Auburn, not withstanding a tainted 2010 "championship".
Speaking of rivals, are you sure you want to be in the SEC? According to Wikipedia, A&M has a 36% winning percentage vs. old rival Arkansas.

(And 32% vs rival Texas. Ouch!)
Arkansas (rivalry dates to 1903) has certainly had our number over the years, but we didn't get to play them in the 90s when we were great and they were pretty bad, so who knows what that record would look like had the series kept going? Ditto with LSU (rivalry dates to 1899), who we actually dominated in the late 80s/early 90s—winning the last five in a row before they canceled the series.
What most people don't know is that A&M was a tiny, all-male, all-military school for the first 80 years of its existence. We had some great teams in the early decades of the 20th Century, but from the start of WWII (when the A&M campus was pretty much a ghost town with all the kids over in the war) until the school changed in the early 70s, we got hammered by everyone.
But since the military aspect ceased to be compulsory, women began to be admitted and the student population was expanded (by about 700% over the course of 40 years—from roughly 7k to almost 50k), A&M athletics has been a bit different (as you can well imagine). I think our modern history compares with just about anyone's we've played (save OU, whose success has been staggering).
Since 1975
Winning Percentage
OU: .744
UT: .706
Aub: .678
A&M: .646
LSU: .634
Ark: .623
MSU: .426
Conference Championships
OU: 17
UT: 9
A&M: 8
Aub: 6
LSU: 5
Ark: 4
MSU: 0
Head-to-Head (A&M wins listed first)
vs. UT: 19-17
vs. LSU: 7-5
vs. Aub: 1-0 (1986 Cotton Bowl)
vs. OU: 6-11
vs. Ark: 6-13
vs. MSU: 0-1 (2000 Indy Bowl)
So if you want to judge Aggie football only by what's happened since 2002, it's easy to say we won't compete in the SEC. If you want to look at our entire history, then you can't ignore all the championships anymore than you can ignore the lack of success against our biggest rivals. If, however, you want to look at our history since we became a bonafide university (as opposed to a tiny military college), then it's a bit harder to make the case that we'll never be any good.
Link to winning % comparisons