How to Fix the 'Black Dot' Unicode Bug Crashing iMessage

[I7guy]

As it seems to be a display problem, I’d expect a fix using an error trap & question mark symbol.
(which might require a core OS specialist 15 mins and Craig 2 mins to mobilize him)

Apple does have a job opening and I can get resumes to Craig in one minute. Takers?
[doublepost=1526036175][/doublepost]

Unicode 10.0 has >136,000 character codepoints plus additional non-character codepoints. That by itself isn't so bad, but some codepoints can (and will) get combined, sometimes even recursively. For example, ö can be expressed both as its own codepoint LATIN SMALL LETTER O WITH DIAERESIS, or by combining the two codepoints for ¨ COMBINING DIAERESIS and o LATIN SMALL LETTER O.

For emoji, it gets a lot more complex than that — there are the skintone modifiers such as EMOJI MODIFIER FITZPATRICK TYPE-3, and various other combining codepoints that let you, say, create a family emoji by combining three different humans.

What you end up with is a practically unlimited amount of permutations, and the best you can do is to hard-limit the amount of recursion allowed. Presumably, Apple and Google already do this in some places.
[doublepost=1526035927][/doublepost]

Sure. My personal schedule is nobody's business, and there are plenty of reasons you wouldn't want someone snooping on where they can find you at what point in time.



Maybe, but that doesn't mean technology shouldn't strive to be better.

But that’s my choice, isn’t it? I totally understand that as do my android endpoints. There are obviously cross platform solutions, but this “just works“.

 

[chucker23n1]

So, if this is the same issue as the Android one, then it's not like the Telugu bug, which was a legitimate character string that was failing,

I don't speak Telugu and thus can't vouch for whether it was legitimate (I believe I read somewhere that it wasn't), but it was a specific combination of five codepoints causing the issue.

but something designed deliberately to crash a renderer by embedding thousands of control characters within a message. But, if that's the case and that you can't crash messages by simply sending a black spot and a pointing emoji to someone, then how does it propagate?

Probably by hitting send before it has fully rendered.
[doublepost=1526038235][/doublepost]

A well designed unicode library should be able to handle it. I think I'm going to investigate which library is the culprit here out of curiosity.

EDIT:
That was quick: http://site.icu-project.org/#TOC-Products-from-Apple

Time to get out the torches & pitchforks.

Yup, Apple definitely uses ICU in some places — it even ships for iOS developers:

In case the NSString support for regular expressions is not sufficient for your needs, a modified version of the libraries from ICU 4.2.1 is included in iOS at the BSD (nonframework) level of the system.

That doesn't necessarily mean that Core Text rendering internally relies on ICU, however.

 

[whooleytoo]

Unicode 10.0 has >136,000 character codepoints plus additional non-character codepoints. That by itself isn't so bad, but some codepoints can (and will) get combined, sometimes even recursively. For example, ö can be expressed both as its own codepoint LATIN SMALL LETTER O WITH DIAERESIS, or by combining the two codepoints for ¨ COMBINING DIAERESIS and o LATIN SMALL LETTER O.

For emoji, it gets a lot more complex than that — there are the skintone modifiers such as EMOJI MODIFIER FITZPATRICK TYPE-3, and various other combining codepoints that let you, say, create a family emoji by combining three different humans.

What you end up with is a practically unlimited amount of permutations, and the best you can do is to hard-limit the amount of recursion allowed. Presumably, Apple and Google already do this in some places.

Thanks for the numbers!

It obviously wouldn't take long to test an individual character or substring (not very many clock cycles) so they could iterate though hundreds of thousands of combinations in relatively little time. Obviously though, if the permutations become too lengthly & complex (e.g. 17 ös followed by a ø will crash the app) then it's a virtually infinite set to test.

In that case, you'd still have to wonder why it's crashing as opposed to simply not displaying correctly. Could they not just (oversimplifying wildly) wrap everything in an exception handler, display a This message cannot be displayed. placeholder instead and move onto the next?

 

[C DM]

Actually it shows that his statement was not that far off.
[doublepost=1526020518][/doublepost]


So the insecure and unencrypted SMS.
I thought security was very important for you iphone users.

It shows that the statement isn't true. Saying that 2+2=5 isn't something that isn't "that far off", it simply isn't true.

 

[chucker23n1]

Thanks for the numbers!

It obviously wouldn't take long to test an individual character or substring (not very many clock cycles) so they could iterate though hundreds of thousands of combinations in relatively little time.

Yup.

(Though even that may be too slow for a unit test.)

Obviously though, if the permutations become too lengthly & complex (e.g. 17 ös followed by a ø will crash the app) then it's a virtually infinite set to test.

Right. The Telugu bug, for example, was a sequence of five characters, so while I'm oversimplifying and I'm sure you could eliminate some combinations for testing, we're talking roughly 136,000^5 permutations, or 4,652587418E25. That's well outside the realm of something you could iteratively test, so you probably want to take an approach like fuzz testing instead.

In that case, you'd still have to wonder why it's crashing as opposed to simply not displaying correctly. Could they not just (oversimplifying wildly) wrap everything in an exception handler, display a This message cannot be displayed. placeholder instead and move onto the next?

Yup. There are probably performance reasons for not doing so.

 

[ROGmaster]

It shows that the statement isn't true. Saying that 2+2=5 isn't something that isn't "that far off", it simply isn't true.

We are not working with exact numbers here.
Yes his statement isn't true but it's also not completely wrong.
[doublepost=1526288156][/doublepost]

What? The “I’ll be there in 10” has to be encrypted? iPhone users understand the limitations of technology and use it appropriately.

Don't make me laugh. That's not what's happening here, you are just trying to establish what matters and what doesn't matter in absolute terms based on your subjectivism.
It's not like SMS conversations are limited to: “I’ll be there in 10” for your point to pe 100% valid.

iMessage is way better and preferable because it's encrypted so it's secure. Which is very important.

Then

Any service that requires me to ask someone to download the app so I can message them isn't worth using

But I thought security is very important so why not download and use an app that offers the same level of security iMessage offers and it's cross-platform? Downloading an app is one of the most trivial things you can do on a smartphone.

 

[I7guy]

We are not working with exact numbers here.
Yes his statement isn't true but it's also not completely wrong.
[doublepost=1526288156][/doublepost]

Don't make me laugh. That's not what's happening here, you are just trying to establish what matters and what doesn't matter in absolute terms based on your subjectivism.
It's not like SMS conversations are limited to: “I’ll be there in 10” for your point to pe 100% valid.

iMessage is way better and preferable because it's encrypted so it's secure. Which is very important.

Then

Any service that requires me to ask someone to download the app so I can message them isn't worth using

But I thought security is very important so why not download and use an app that offers the same level of security iMessage offers and it's cross-platform? Downloading an app is one of the most trivial things you can do on a smartphone.

Why exactly are we having this conversation?SMS is native, unencrypted and easy to hack and yet is the lowest common denominator between here and android. My choice. Security is important and doesn’t mezn I don’t mind telling the world when I want..I’ll be there at ten. More private conversations, I’ll use iMessage.

It’s not really a problem as my android friends are slowly converting to iPhone.

 

[ROGmaster]

So private conversations are a no no with Android users.
And what would you advise them to do? buy iphones in order to have private conversations with you?

It’s not really a problem as my android friends are slowly converting to iPhone.

Well that is just splendid, you are a lucky guy.

Most of my friends with iphones don't care for iMessage at all(like most iphone users in Europe for that matter), they use it as a SMS messaging app and prefer cross platform alternatives like Whatsapp, Telegram etc. Isn't that funny?

 

[C DM]

We are not working with exact numbers here.
Yes his statement isn't true but it's also not completely wrong.
[doublepost=1526288156][/doublepost]

Don't make me laugh. That's not what's happening here, you are just trying to establish what matters and what doesn't matter in absolute terms based on your subjectivism.
It's not like SMS conversations are limited to: “I’ll be there in 10” for your point to pe 100% valid.

iMessage is way better and preferable because it's encrypted so it's secure. Which is very important.

Then

Any service that requires me to ask someone to download the app so I can message them isn't worth using

But I thought security is very important so why not download and use an app that offers the same level of security iMessage offers and it's cross-platform? Downloading an app is one of the most trivial things you can do on a smartphone.

Isn't true already undermines it and anything else that might be said.

 

[I7guy]

So private conversations are a no no with Android users.
And what would you advise them to do? buy iphones in order to have private conversations with you?


Well that is just splendid, you are a lucky guy.

Most of my friends with iphones don't care for iMessage at all(like most iphone users in Europe for that matter), they use it as a SMS messaging app and prefer cross platform alternatives like Whatsapp, Telegram etc. Isn't that funny?

That’s correct. SMS is not used to send confidential or personal information. Typically getting around the platform differences requires a phone call.

Sorry you are having a tough time with this, but I hate playing “where’s my message app”.

 

[TheSkywalker77]

Maybe Apple could spend some of their money fixing these issues instead of putting in useless features nobody uses...

Looking at you Animoji.

 

[I7guy]

Maybe Apple could spend some of their money fixing these issues instead of putting in useless features nobody uses...

Looking at you Animoji.

So with 100,000 employees Apple can’t fix bugs and add emojis/Animoji, which is the showcase of their technology?

 

[gtg465x]

Jesus H. Christ. I'm not familiar with how unicode characters are decoded, but can a dev at Apple please put the logic in a try catch block so there isn't a new character or character combo discovered that crashes the app every other month?

 

[chucker23n1]

Jesus H. Christ. I'm not familiar with how unicode characters are decoded, but can a dev at Apple please put the logic in a try catch block so there isn't a new character or character combo discovered that crashes the app every other month?

No. C doesn't have try/catch, and adding exception handling would probably cause too much of a performance hit.

 

[gtg465x]

No. C doesn't have try/catch, and adding exception handling would probably cause too much of a performance hit.

I guess it depends on where the code is. The higher level libraries and frameworks in iOS are written in ObjC, which does support exception handling. I know it's usually considered bad practice to use try catch in many languages if there's a way to do checks to avoid the exceptions altogether, and it can cause performance hits, but these "another character causing crashes" bugs are getting ridiculously common. Surely something can be done, even if it's not try catch blocks.

 

[chucker23n1]

I guess it depends on where the code is. The higher level libraries and frameworks in iOS are written in ObjC, which does support exception handling.

Yes, and Core Text is not high-level, for performance and portability reasons.

these "another character causing crashes" bugs are getting ridiculously common. Surely something can be done, even if it's not try catch blocks.

Maybe something can be done, but I doubt incompetence is the reason these issues keep coming up — Unicode is, in practice, quite hard to handle.

 

[ROGmaster]

That’s correct. SMS is not used to send confidential or personal information. Typically getting around the platform differences requires a phone call.

LoL that's not what I've asked. You still haven't answered what happens when you don't have I’ll be there at ten type conversations with android user but a confidential and personal ones.
If I don't own a phone that has iMessage I can't have a personal and private conversation of what? What's really the deal here?
It seems that this hole iMessage argument is so thin you constantly have to dance around the subject.

Sorry you are having a tough time with this, but I hate playing “where’s my message app”.

What does this even mean?
[doublepost=1526361922][/doublepost]

Isn't true already undermines it and anything else that might be said.

I don't see how it does that.

 

[I7guy]

LoL that's not what I've asked. You still haven't answered what happens when you don't have I’ll be there at ten type conversations with android user but a confidential and personal ones.
If I don't own a phone that has iMessage I can't have a personal and private conversation of what? What's really the deal here?
It seems that this hole iMessage argument is so thin you constantly have to dance around the subject.



What does this even mean?
[doublepost=1526361922][/doublepost]
I don't see how it does that.

You know what. This works for me, that’s all you need to know. This started out with some generalized statement of iPhone users and security. Should have called you on that immediately.

 

[d123]

iPhone 5 with iOS 6 here...immune to any and all of these issues haha

But on the downside, you're still using an iPhone 5 in 2018...

 

[C DM]

LoL that's not what I've asked. You still haven't answered what happens when you don't have I’ll be there at ten type conversations with android user but a confidential and personal ones.
If I don't own a phone that has iMessage I can't have a personal and private conversation of what? What's really the deal here?
It seems that this hole iMessage argument is so thin you constantly have to dance around the subject.



What does this even mean?
[doublepost=1526361922][/doublepost]
I don't see how it does that.

It does that in the same sense that if someone makes it a specific point to say and believe that 2+2=5 you can't really just go by what they say given that it can easily not be true (given that the one thing they tried to really say was already incorrect).

 

[Avenged110]

But on the downside, you're still using an iPhone 5 in 2018...

Eh, it works just as well now as when I bought it, which is to say really well, actually. Other than the camera, they've added nothing compelling for me while destroying the software.

 

[TyWahn]

What happens to the phones that don't have 3d Touch - like an iPhone 6 user. :-(

 

[ROGmaster]

It does that in the same sense that if someone makes it a specific point to say and believe that 2+2=5 you can't really just go by what they say given that it can easily not be true (given that the one thing they tried to really say was already incorrect).

I disagree and this is not a 2+2=5 situation.