Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

MacRumors

macrumors bot
Original poster
Apr 12, 2001
68,589
39,460



twitterlogo.jpg
Twitter is suggesting that all Twitter users update their passwords following a glitch that exposed some passwords in plaintext on its internal network.

As outlined in a blog post, Twitter says that it recently found a bug that "stored passwords unmasked in an internal log." The bug was fixed, and an internal investigation shows that there was no breach or misuse.
We mask passwords through a process called hashing using a function known as bcrypt, which replaces the actual password with a random set of numbers and letters that are stored in Twitter's system. This allows our systems to validate your account credentials without revealing your password. This is an industry standard.

Due to a bug, passwords were written to an internal log before completing the hashing process. We found this error ourselves, removed the passwords, and are implementing plans to prevent this bug from happening again.
Despite the fact that no one appears to have accessed the plaintext passwords, Twitter is recommending that all users "consider" changing their passwords "out of an abundance of caution" both on Twitter and on any other site where the same password was used.

If you're a Twitter user, you can change your password on the web by accessing your Twitter settings and selecting the password option. You will need to enter a current password and then choose a new one. In the Twitter iOS app, you'll need to sign out to initiate a password change.

Using a unique password for every login is the best way to make sure you stay secure in the event of a data breach, something best managed with an app like 1Password or LastPass.

Twitter is recommending users choose a unique, strong password and then protect their accounts with two factor authentication.

Article Link: Twitter Recommends Changing Your Password Following Plaintext Exposure Glitch
 
Got an email from GitHub saying the same thing. I guess they're using the same library somewhere.

Hi there,

During the course of regular auditing, GitHub discovered that a recently introduced bug exposed a small number of users’ passwords to our internal logging system, including yours. We have corrected this, but you'll need to reset your password to regain access to your account.

GitHub stores user passwords with secure cryptographic hashes (bcrypt). However, this recently introduced bug resulted in our secure internal logs recording plaintext user passwords when users initiated a password reset. Rest assured, these passwords were not accessible to the public or other GitHub users at any time. Additionally, they were not accessible to the majority of GitHub staff and we have determined that it is very unlikely that any GitHub staff accessed these logs. GitHub does not intentionally store passwords in plaintext format. Instead, we use modern cryptographic methods to ensure passwords are stored securely in production. To note, GitHub has not been hacked or compromised in any way.

You can regain access to your account by resetting your password using the link below::

https://github.com/password_reset

If you have any lingering questions or concerns about this, don't hesitate to let us know. You can reach us by emailing support@github.com or by using our contact form:

https://github.com/contact

Thanks,

GitHub Support

My question is why the plaintext password is even sent to them for password resets. There's zero reason for that. Shouldn't it just be validated and hashed by the webpage's script before sending?
 
Last edited:
This is inexcusable. Which developer had the bright idea to store passwords to a log file? That should never happen, ever. There's no reason for it.


I hope that means they fired the person responsible, or sent them back to primary school to learn basic common sense.
I'll bet the person responsible wasn't hired by anyone, just an open source contributor. And I can't imagine what bug put passwords into the log file, but definitely nobody meant to do that.
 
  • Like
Reactions: benface
My question is why the plaintext password is even sent to them for password resets. There's zero reason for that. Shouldn't it just be validated and hashed by the webpage's script before sending?

There is no hashing done by web page scripts unless the site you are on uses Javascript for both the front end and back end (such as with React, node.js, etc.). Twitter is not one of those sites. However, values are encrypted over the line due to the SSL certificate (why you see HTTPS in the browser), but those values are decrypted on the server. The web server has to handle them somehow, and certainly can't validate anything, create new records (tweets, users, etc.) or update anything (such as your user profile) if it's just garbled encryption. It is not plaintext over the line.

This is inexcusable. Which developer had the bright idea to store passwords to a log file? That should never happen, ever. There's no reason for it.

Twitter was built on Ruby on Rails. It has, I believe, since migrated to another platform, but many of the concepts still remain, regardless of framework used. In Rails, for example, everything is logged - all parameters sent from a form (login info, new tweet message, profile settings, etc.) go to the log file, as well as database transactions and manual log messages.

In real world applications, regardless of programming framework used, logging either goes to an actual file on the drive of a server, or to a drain that feeds the log line-by-line to a service (so it can be searched or you can receive alerts on errors, etc.). There are also multiple levels of logging depending on what needs priority - everything from fatal and error messages that have higher priority, to info and debug messages for general system events. Things like this would have been an info message with parameters received from a client, POSTing to a particular endpoint. Those basic info and debug messages can be omitted from production logs, which does make debugging errors more difficult, but is often done for security purposes (the "use a sledgehammer to hammer in a nail" approach).

In most cases, the application framework employs sanitizers to mask sensitive parameters from being sent to the log (i.e. passwords, credit card numbers, social security numbers, etc.). This happens in a configuration file that isn't touched often, if ever, after the application is deployed on a website. Additionally, masking sensitive parameters occurs in production but not always in local development, since building, updating, or fixing features requires a higher level of knowledge of what's going on vs. once something has gone live.

My guess is they either accidentally turned off the sanitizer, changed the password field name or are using an alias field name to prevent bots (most likely case), or never masked it in the first place and decreased the log level for debugging purposes. It's a simple mistake that isn't easily apparent.

In any case, this happened on their end, they noticed it, and they let us know. There's no indication that anything has actually been accessed. And, even still, with most accounts using 2FA, most users staying perpetually logged in, and with API keys being how external applications authenticate to your Twitter account (not with your password), the likelihood of your password even showing up in the log is very slim anyway. I'm not saying you shouldn't change your password (because you absolutely should), but this sounds much scarier than it actually is.
 
There is no hashing done by web page scripts unless the site you are on uses Javascript for both the front end and back end (such as with React, node.js, etc.). Twitter is not one of those sites. However, values are encrypted over the line due to the SSL certificate (why you see HTTPS in the browser), but those values are decrypted on the server. The web server has to handle them somehow, and certainly can't validate anything, create new records (tweets, users, etc.) or update anything (such as your user profile) if it's just garbled encryption. It is not plaintext over the line.
No, where does the website need the plaintext password instead of the hashed one? They just hash it then throw it away ASAP. Any validation can be done with the hashed one. Nearly every site uses JS for frontend, not that you need JS to do hashing.

Say I write a browser extension that hashes all passwords sent in forms before sending them, always using the same salt. With no other changes, that would work fine with every website. Heck, maybe I should do that.
 
Last edited:
Got an email from GitHub saying the same thing. I guess they're using the same library somewhere.



My question is why the plaintext password is even sent to them for password resets. There's zero reason for that. Shouldn't it just be validated and hashed by the webpage's script before sending?

No, client-side password hashing offers no security benefit. Furthermore, relying on only client-side hashing comes at a security detriment, as the hashed password sent over the wire is now, in effect, the user's password.
 
This is inexcusable. Which developer had the bright idea to store passwords to a log file? That should never happen, ever. There's no reason for it.


I hope that means they fired the person responsible, or sent them back to primary school to learn basic common sense.


Wow, you don't do much programming at large scale do you? Logging is a necessity and sometimes things like this happen. It's quite common and that's just how it goes. I highly doubt it was conscious decision to log passwords in plain text. Furthermore, firing a developer for something like this hurts the company in a larger way. It only scares others into not reporting issues. Twitter has done a great job of handling this in a blameless manner, and I am certain their technical post-mortem will address the root cause.
[doublepost=1525391354][/doublepost]
No, where does the website need the plaintext password instead of the hashed one? They just hash it then throw it away ASAP. Any validation can be done with the hashed one. Nearly every site uses JS for frontend, not that you need JS to do hashing.

Say I write a browser extension that hashes all passwords sent in forms before sending them, always using the same salt. With no other changes, that would work fine with every website. Heck, maybe I should do that.


What? Just no...

Client-side hashing does nothing... The website receiving your hash has no context to know you have already hashed it. It sees a string of text that it knows as your password. You are in the same boat should it be compromised...

Not sure what you mean by "any validation can be done with the hashed one?" Hashing is by nature a one-way operation. You can't validate password complexity based on a hash...
 
Got an email from GitHub saying the same thing. I guess they're using the same library somewhere.



My question is why the plaintext password is even sent to them for password resets. There's zero reason for that. Shouldn't it just be validated and hashed by the webpage's script before sending?

Most sites do the hashing server-side (not a problem if you’re using SSL) but the logic for hashing passwords server-side is so simple it shouldn’t require third-party libraries and is so simple that I don’t understand at which point this could possibly happen. Unless for whatever reason you’re arbitrarily dumping all POST data...
 
What? Just no...

Client-side hashing does nothing... The website receiving your hash has no context to know you have already hashed it. It sees a string of text that it knows as your password. You are in the same boat should it be compromised...

Not sure what you mean by "any validation can be done with the hashed one?" Hashing is by nature a one-way operation. You can't validate password complexity based on a hash...
Client-side hashing protects my plaintext password. If the hashed version were revealed, yes, my account on that site would be temporarily insecure, but at least there'd be no risk of anyone seeing my password (bad for various reasons).

When I said any validation can be done, I meant what guy I replied to was mentioning for auth or anything else involving equality-checking, i.e. everything they need to do after you sign up. For checking password complexity during signup, you can easily do that with a client-side script.
[doublepost=1525391782][/doublepost]
Most sites do the hashing server-side (not a problem if you’re using SSL) but the logic for hashing passwords server-side is so simple it shouldn’t require third-party libraries and is so simple that I don’t understand at which point this could possibly happen. Unless for whatever reason you’re arbitrarily dumping all POST data...
But it is evidently a problem. IDK what they're using the lib for. Sounds like some traffic logging thing. I'm sure this happens on plenty of lesser known sites, and they don't tell you, so we all might as well consider our old passwords stolen. In general, security credentials shouldn't be needlessly sent (edit: plaintext or transformed in a reversible way) between parties, and this is one reason why.
 
Last edited:
No, where does the website need the plaintext password instead of the hashed one? They just hash it then throw it away ASAP. Any validation can be done with the hashed one. Nearly every site uses JS for frontend, not that you need JS to do hashing.

Say I write a browser extension that hashes all passwords sent in forms before sending them, always using the same salt. With no other changes, that would work fine with every website. Heck, maybe I should do that.

Huh? How else would you do hashing in the browser? They’d lose half their user base if they required people to download an extension purely to client-side hash their passwords in order to log in.

Plus, if you hash client-side, you would have to expose your salts publicly, defeating the purpose of them.
 
Huh? How else would you do hashing in the browser? They’d lose half their user base if they required people to download an extension purely to client-side hash their passwords in order to log in.

Plus, if you hash client-side, you would have to expose your salts publicly, defeating the purpose of them.
I just meant that you can use any client-side script to hash. JS seems to be what everything supports today. A decade ago, people used Flash and Java. Netflix used to require Silverlight a few years ago. Google is pushing Dart now.

Client chooses salt, sends it with hash (actually bcrypt in particular combines them). Server could store exactly that, but actually the server should also re-hash it with its own salt.
 
I just meant that you can use any client-side script to hash. JS seems to be what everything supports. A decade ago, people used Flash and Java. Netflix used to require Silverlight.

Client chooses salt, sends it with hash (actually bcrypt in particular combines them). Server could store exactly that, but actually the server should just re-hash it itself with its own salt.

I suppose you could theoretically hash it once with JS (like hash the plain text password) and then the server would hash the hash? Is that what you mean? Because you *must* hash server-side or you’re effectively storing passwords in plain text, even though they are a hash.
 
I suppose you could theoretically hash it once with JS (like hash the plain text password) and then the server would hash the hash? Is that what you mean? Because you *must* hash server-side or you’re effectively storing passwords in plain text, even though they are a hash.
Yes. Every party storing or sending a password should hash it first.
 
It’s about time we confined passwords to the history books. Touch ID or Face ID or whatever should replace passwords.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.