Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Special Characters in SMS turn them shorter (70 characters). Old issue, never solved.

luisadastwin

luisadastwin

macrumors regular
Original poster
Oct 9, 2010
175
2
Antas - Porto - Portugal - Europe
This is only an issue for non-English speaking countries... That's why this is not a big deal for the North-Americans that (comprehensively) sometimes simply don't understand what are we talking about...

In any mobile Phone (iPhone included), when you write a SMS using special characters like "ç","á","é","í","ó","ú","ñ","ä", etc, the SMS are limited to only 70 characters instead of 160 characters because a extended encoding is necessary! So, you pay 3x the price of an SMS if you write 160 characters (it will send 3 SMS: 70+70+20). Besides that, the risk of the SMS not being delivered is higher because it is actually sending 3 SMS at once and sometimes it gets broken in the way.

So, to solve this issue, there is a feature in any Nokia mobile phone almost for 10 YEARS (!!!) called: "SMS Extended Character Set": ON/OFF

If turned ON, it uses extended character set with 70 characters per SMS and the destination Phone gets all the special characters: "ç","á","é","í","ó","ú","ñ","ä"

If it is turned OFF: the phone allows to write special characters, with 160 characters per SMS, but automatically converts the special characters to their basic equivalents "c","a","e","i","o","u","n","a" when sending the SMS.

... It is amazing how Apple is 10 years behind other mobile manufacturers in this matter... Maybe because Nokia is originally from Finland (where they use special characters and experienced the issue) while Apple is from the USA, where they naturally (and sadly) don't recognize this as a issue.

Imagine if you send 1000 SMS in a week and you have to pay them (many people pay them)... You have to be very careful not to introduce any special character otherwise you pay 3x the money!

It is a good thing OS4 has now the Character counter (that turns to 70 when writing a special character) but still is very very annoying because iPhone has auto-complete and auto-correct and always suggests the "correct" word! One drastic example is the Portuguese word "abraço" (it means "hug")! It is how we end almost all the SMS we send and when you write it like "abraco" (without the "ç"), iPhone corrects automatically to "abraço" every time! and here we go, correcting it again and again... arrgghhh....

Well, a simple feature that would make life easier for all non-English speakers around the world, don't you think?

Any ideas about this?

Thank you!
 
That is really interesting.

Also, the character gets even weirder as german Umlauts (ä ö ü) don't change the SMS counter to x / 70.
(i was wondering whether i just hadn't noticed this before, but german seems to work perfectly fine)

What's even more random is:

While the accent aigu on the A (á) triggers it, the grave (à) doesnt.
On the E, however, it's the other way around. é keeps it at x/160, è changes to x/70.

'ñ' works, actually, even the 'nordic' å or ø do work.
The SMS character set is really really random :/
 
I am sorry, I was not precise.

You are correct about the umlaut (ä) and ñ.

In fact, it seems a little random but it is not! :)

It is very well defined.... Some of the Umlauts and the "é" are part of the Standard GSM Characters set ... Others do not make part of that set...

But, in other hand, the ã, á, í, ó, è, etc.. are not part of the Standard GSM Characters set...

Anyway, iPhone should convert those characters outside the Standard GSM Characters set to their Standard GSM Characters equivalents...

So: The "ä" should be the same in the destination phone, but the "è" or the "ë" should be converted to the basic "e" avoiding the 70 characters limit...


Lool at this: http://www.mediaburst.co.uk/blog/the-gsm-character-set/

It shows what characters are part of Standard GSM Characters set!
And this is CASE SENSITIVE! Any other special Character different then these will be considered outside the standard GSM character set!

Standard: è, é, ù, ì, ò, Ç (upper-case!), ø, Å, å, æ, ß, É, Ä, Ö, Ñ, Ü, ä, ö, ñ, ü, à
 
I, for one, was not aware of this particular issue, but it DOES demonstrate a stubbornness on the part of Apple when it comes to implementing simple things that have been available on cell phones dating back to some of the very first phones.
As much as i love my iPhone, there are some features that I had on my first Nokia that I would like to see in the IOS.
 
Mliii, Geez...!

You just took the words right of my mouth! (I don't know if this applies in English but it's a translation of a very used expression in Portuguese that means that "you said exactly what I was about to say" :))

That's it..! I really love my iPhone but it still misses so many basic features that my ten year old Nokias had... It is a real shame... The alerts are another one... I can only define an alert 5 minutes before the event or 15 minutes, 30 minutes, 1 hour, 2 hours, 1 day... Ridiculous!!!!! Why can't I define an alert 6 hours before??? Or 2 minutes?? Whatever!!!! Ridiculous... But that's another thread :p
 
It would be my understanding that when the amount of characters goes up, it would be caused by the character being entered is actually entered in an HTML type format.

This is just a guess as I've seen this before.
I know on my Nexus One that if you put in 'ç' it actually converts the message to a multimedia message.

Saying that though, the HTML for 'ç' is '&#231 ;' so it's not really that many more characters. But then from Apples side it could be wrapped in more code.

I am just guessing to be honest and trying to give some kind of explanation to what could possibly be causing it.
 
Thank you for you contribution kAoTiX, but, in this case this is a perfectly known and identified behaviour and it has nothing to do with HTML (except "Nexus One" as I will refer in a minute).

The iPhone, just like a regular and perfectly normal mobile phone, uses a universally standardized communications protocol: the "SMS" (Short Message Service) and the number of bytes for each SMS is only 140 bytes. This limitation of 140 bytes has never been changed for MANY MANY years and that's why it is so small...

So, if you have a maximum of 140 bytes, you have a maximum of 1120 bits:

1 byte = 8 bits.
140 bytes = 140*8 bits = 1120 bits

This way we know that the MAXIMUM number of bits per SMS is 1120!


So, if you ONLY use characters from the Standard GSM Characters set (or also refereed "GSM 7 bit default alphabet"), it will only need 7 bit to encode each character!

Take a look at this table which is the Standard GSM Characters / GSM 7 bit default alphabet: http://en.wikipedia.org/wiki/GSM_03.38

This way, the phone can support a total of :

1120 bits / 7 bits = 160 characters for each SMS:! ;)


Now, if you use ANY character rather than the characters shown in the referred table (the Standard GSM Characters set), because of THAT character, the mobile phone will have to use 16 bit encoding, using 16 bit to encode each of the characters! (instead of only 7 bit!)

So, if it consuming 16 bits for each character:

1120 bits / 16 bits = 70 characters for each SMS:! :(

So, if I send a SMS with 160 characters with JUST 1 (ONE) special character, it will split the text into 3 (three) SMS: 70+70+20 = 160 characters, making it more expensive and with higher probability of failing delivery (since it is a 3-linked-SMS).

But Nokia, in an attempt to please its customers, implemented a feature than automatically converts the special characters in to their standard equivalents, leaving it with a 160 characters maximum, the same way :cool:

It is a basic features that the Nokia phone (and other brands) used for a LONG LONG LONG time because it is very useful. And iPhone should also have this feature that is so, so simple!


The "Nexus One" case you present it is even more ridiculous than the iPhone in this matter because it is avoiding the SMS protocol (using MMS protocol instead of 16-bit encoding SMS protocol) when you insert a character "outside" the Standard GSM Characters set, making it even more expensive! :confused: :confused: :confused: :confused:



kAoTiX said:
It would be my understanding that when the amount of characters goes up, it would be caused by the character being entered is actually entered in an HTML type format.

This is just a guess as I've seen this before.
I know on my Nexus One that if you put in 'ç' it actually converts the message to a multimedia message.

Saying that though, the HTML for 'ç' is 'ç ;' so it's not really that many more characters. But then from Apples side it could be wrapped in more code.

I am just guessing to be honest and trying to give some kind of explanation to what could possibly be causing it.
Click to expand...
 
luisadastwin said:
But Nokia, in an attempt to please its customers, implemented a feature than automatically converts the special characters in to their standard equivalents, leaving it with a 160 characters maximum, the same way :cool:
Click to expand...

Haven't used a Nokia in many years: what is the 'standard equivalent'? Does it turn a ç in a c? If that's the case...why you waste the effort typing it in the first place? Plus the message loses meaning.
 
If you read my very first post, you wouldn't post this one :rolleyes:

You are right in one thing:

The standard equivalents: ç = c, á = a, é = e, etc...

But why don't I write the "c" instead of "ç"???... well... It is very simple (as I said in the first post):

I write a LOT of SMS like many other people and to write them MUCH MUCH faster I use "auto-complete" or "Intelligent SMS writing".

If you notice, the iPhone can predict or correct a misspelled word!

The example I gave was "abraço" which is "hug" in Portuguese and I write it in 90% of the SMS I send.

So if I write "abraco" (with "c") instead of "abraço" (with "ç"), the iPhone will auto-correct to "abraço" again and again! and this happens with MANY MANY other words during the SMS writing which is VERY annoying has you can imagine. :(

And, by the way, everybody here understands that "abraco" means "abraço" (hug) or "arvore" means "árvore" (tree), or "orgao" means "órgão" (organ) ... etc.... etc...

It is rare the loss of meaning because almost every words in Portuguese don't change meaning when you loose the "´", "~", "ç", etc....

Other thing.. all the words in English that end with "..tion" in Portuguese is "..ção" .. like "situation" --> "situação", "education" --> "educação"... so imagine how many words I have to re-correct in the iPhone :( :( :(

Loosing the special characters and turning into the standard equivalents is perfectly normal in SMS sendin in Portugal. Its common sense here and very easy to interpret. Actually, I the Portuguese kids are already writing the words at school without accents or special characters because of the SMS sending! LOOOOOOL



sammich said:
Haven't used a Nokia in many years: what is the 'standard equivalent'? Does it turn a ç in a c? If that's the case...why you waste the effort typing it in the first place? Plus the message loses meaning.
Click to expand...
 
sammich said:
Haven't used a Nokia in many years: what is the 'standard equivalent'? Does it turn a ç in a c? If that's the case...why you waste the effort typing it in the first place? Plus the message loses meaning.
Click to expand...
Agreed.

This isn't an issue, and Nokia's fix isn't a good idea.
 
Sorry :eek: I'm not usually one for skipping reading, I guess it's getting late here.

Fair enough about that I guess. Apple could very easily add the switch but there's a possibility that they like keeping things in their original/intended format.

I presume you've tried pressing the 'x' when you finish typing 'abraco' (before pressing space) when it offers the correction of 'abraço'? That should teach it to stop autocorrecting it. Sure, you can't do this to every word, and it'll mean that the autocorrect won't correct in other apps like Mail/Web/Notes etc.
 
I am sorry, but where are you from? Did you experience this to say it is not a good idea? :p

And, by the way, It is NOT a Nokia FIX! It is an OPTION! A FEATURE that you CAN, if YOU WISH, activate it! Otherwise, it remains the same? Get it? :D

As I also said, loosing the special characters and turning into the standard equivalents is perfectly normal in SMS sending in Portugal. Its common sense here and very easy to interpret. And you get ALL the 160 characters! That is important, right?


:)

Daveoc64 said:
Agreed.
This isn't an issue, and Nokia's fix isn't a good idea.
Click to expand...
 
Exactly Sammich!!!!

That's it!!!

Now, when I write e-mails, it does not correct some of the words, which is less interesting when you write a Professional e-mail :D:D:D:D




sammich said:
Sorry :eek: I'm not usually one for skipping reading, I guess it's getting late here.

Fair enough about that I guess. Apple could very easily add the switch but there's a possibility that they like keeping things in their original/intended format.

I presume you've tried pressing the 'x' when you finish typing 'abraco' (before pressing space) when it offers the correction of 'abraço'? That should teach it to stop autocorrecting it. Sure, you can't do this to every word, and it'll mean that the autocorrect won't correct in other apps like Mail/Web/Notes etc.
Click to expand...
 
luisadastwin said:
I am sorry, but where are you from? Did you experience this to say it is not a good idea? :p

And, by the way, It is NOT a Nokia FIX! It is an OPTION! A FEATURE that you CAN, if YOU WISH, activate it! Otherwise, it remains the same? Get it? :D

Has I also said, loosing the special characters and turning into the standard equivalents is perfectly normal in SMS sending in Portugal. Its common sense here and very easy to interpret. And you get ALL the 160 characters! That is important, right?


:)
Click to expand...

I've seen the feature, and I've used it.

It's stupid because it makes the process of sending a message inconsistent. If you include a particular character in a message, that character should be sent in the message. If you don't want to use those characters for whatever reason, don't put them in your messages!
 
Come on man... You can't be serious... Did you read all the thread? Did you see the last post about auto-correcting...? :confused::confused:

Come on... please I am asking you to read this carefully:

If you thing it is a "STUPID" to use the feature, just don't use it! If you want the special characters to be sent as you say, you simply let the feature be ENABLED, which is by DEFAULT and you don't have to do a thing! If you want to use 160 characters, losing the special characters to their basic equivalents, just turn the FEATURE OFF as an OPTION...! It's not obligatory!

Like I said in the first place:
"There is a feature in any Nokia mobile phone called: "SMS Extended Character Set": ON/OFF

If turned ON, it uses extended character set with 70 characters per SMS and the destination Phone gets all the desired special characters: "ç","á","è","í","ó","ú","â","ã"

If it is turned OFF: the phone allows to write special characters, with 160 characters per SMS (this is the advantage and the PURPOSE of the feature), but automatically converts the special characters to their basic equivalents "c","a","e","i","o","u","a","a" when sending the SMS."


Daveoc64 said:
I've seen the feature, and I've used it.

It's stupid because it makes the process of sending a message inconsistent. If you include a particular character in a message, that character should be sent in the message. If you don't want to use those characters for whatever reason, don't put them in your messages!
Click to expand...
 
I realize this isn't the hacks forum, but if this really is a huge issue for anybody, you might want to consider jailbreaking. BiteSMS has that feature, and it's smart enough to only switch your characters if the message will be over 70 characters otherwise, for short messages it leaves them intact.

I have no use for it personally, but it definitely seems like an easy feature for Apple to implement. Hopefully they'll get around to it soon.
 
Hmmm.. Very interesting this BiteSMS app phixionalninja, thanks!

I actually have JB my iPhone but I think I realized that BiteSMS app is a different app... I was hoping to find a solution that would keep me using iPhone's native SMS app, tweaking it ...

So, BiteSMS has its own SMS app, right? If not, I'll run and buy it :)

Thanks again.

Luis


phixionalninja said:
I realize this isn't the hacks forum, but if this really is a huge issue for anybody, you might want to consider jailbreaking. BiteSMS has that feature, and it's smart enough to only switch your characters if the message will be over 70 characters otherwise, for short messages it leaves them intact.

I have no use for it personally, but it definitely seems like an easy feature for Apple to implement. Hopefully they'll get around to it soon.
Click to expand...
 
luisadastwin said:
Hmmm.. Very interesting this BiteSMS app phixionalninja, thanks!

I actually have JB my iPhone but I think I realized that BiteSMS app is a different app... I was hoping to find a solution that would keep me using iPhone's native SMS app, tweaking it ...

So, BiteSMS has its own SMS app, right? If not, I'll run and buy it :)

Thanks again.

Luis
Click to expand...

It is its own app, but that really isn't a bad thing. It looks and feels very similar to the native app, and the two don't interfere with each other in any way (so any texts sent and received in either one will appear in the other, and if you wanted to, you could switch back and forth constantly and never have any trouble). I just have BiteSMS in my dock and the native app hidden off in my folder for native apps I don't need (along with Stocks, Compass, etc).

Take a scroll down this crazy list of features, it's absolutely worth your time to try it, especially if you're already jailbroken. It's free to try for 10 days or so, and after that it's still free, but with ads at the bottom, or you can pay for a license and have it ad free (I paid, it's worth the money).
 
WOOOOOOAAAAAAHH!!!!!

That's AMAZING!!!!!!!!

Besides what I was looking for, the accents and diacritics feature: http://www.bitesms.com/main/more_features#accents-diacritics, even not in another ten years Apple will have that really CRAZY list of FANTASTIC features in SMS like BiteSMS!!! It's really amazing how it covers all my needs (and needs I didn't realize I had until now :p )!!!

Well, I was always not keen to have different apps rather than the tweaked native ones, but in this case... I really have to think twice and just maybe I will buy it :p

These guys seemed German to me (because of the BiteSMS credit only working in the German network) but actually the contact address is from Australia :p Who would have figured that the accents and diacritics issue would be solved by English-speaking country :p hehe!

Thank you phixionalninja, maybe this app will change my life! :cool: :D


phixionalninja said:
It is its own app, but that really isn't a bad thing. It looks and feels very similar to the native app, and the two don't interfere with each other in any way (so any texts sent and received in either one will appear in the other, and if you wanted to, you could switch back and forth constantly and never have any trouble). I just have BiteSMS in my dock and the native app hidden off in my folder for native apps I don't need (along with Stocks, Compass, etc).

Take a scroll down this crazy list of features, it's absolutely worth your time to try it, especially if you're already jailbroken. It's free to try for 10 days or so, and after that it's still free, but with ads at the bottom, or you can pay for a license and have it ad free (I paid, it's worth the money).
Click to expand...
 
phixionalninja,

I am using BiteSMS now... I'm loving it.... And what about the QuickReply!?!? WOOOW!!

Now I had a small problem... How to "Erase" the "wrong learned words" from the iPhone...? :p

Like I said, the iPhone learned some words without the accents and that is not good when writing an email...

Thanks to volkspost, he told me the solution:

Go to:

/private/var/mobile/Library/Keyboard

In there I found one file, starting with my language setting then "dynamic-text.dat": pt_PT-dynamic-text (Portuguese system).

The file can be edited with a plain text editor with all the words the iPhone learned! I just searched for the "wrong" words and just deleted it!

Fantastic! :D :D :D

Thanks phixionalninja and volkspost
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back