From
this article, it seems that the standard ssh protocol has some issues with latencies on high speed connections. Maybe you can try to compile the patched OpenSSH package on your client and/or server.
And one more question - what happens if you run two or more sftp sessions in parallel? How does the total download speed change?