If you just moved an existing SIM over to an iphone6/6+ it may not be capable.
I used the SIM that shipped with the iPhone 6. Do I need to replace it?
See the bolded text in my prior post.
Since you didn't move an existing SIM, then what I said likely does not apply.
BTW, what did AT&T say when you called them up to ask what's going on? I'd guess something isn't correctly provisioned on your account since you have this problem and are using the SIM that shipped with the phone.
For what it's worth I hadn't really looked before, but did a couple test calls and noticed my LTE indicator remains when on an active call. I'm using the SIM from my 5S and am in the ATL market.