PDA

View Full Version : A faster way to speed up Mail.app


MacBytes
Mar 1, 2007, 08:28 AM
http://www.macbytes.com/images/bytessig.gif (http://www.macbytes.com)

Category: Tips and How To's
Link: A faster way to speed up Mail.app (http://www.macbytes.com/link.php?sid=20070301092817)
Description:: As everyone knows, it is possible to get quite a speed boost out of Mail.app by stripping all the bloat out of its Envelope index, an SQLite database Mail uses to store senders, recipients, subjects and so on. In the past, I suggested that quitting Mail, deleting the Envelope file and restarting Mail would force a rebuild that produces a leaner, faster email experience. In October last year Dallas noticed a faster way to get the same result using three Terminal commands.

Posted on MacBytes.com (http://www.macbytes.com)
Approved by Mudbug

Mudbug
Mar 1, 2007, 08:29 AM
I just did this myself, and to say there's a speed difference may be the understatement of the decade. Mail.app is easily twice the speed it was before running this.

MacBoobsPro
Mar 1, 2007, 08:31 AM
What is it deleting exactly?

What are the consequences?

mkrishnan
Mar 1, 2007, 08:32 AM
This doesn't interrupt threading of subjects or anything, does it? I have to try this when I get home!

MacBoobsPro
Mar 1, 2007, 08:33 AM
This doesn't interrupt threading of subjects or anything, does it? I have to try this when I get home!

My plan exactly but I need to know what it is actually doing incase something goes wrong.

dr_lha
Mar 1, 2007, 09:14 AM
What is it deleting exactly?
Its not deleting anything, its optimizing the database

What are the consequences?
Faster Mail.app

redwin11
Mar 1, 2007, 09:15 AM
Impressive!
The sluggishness has disappeared, and even opening 5-10Mb attachments is quite nippy.

MacBoobsPro
Mar 1, 2007, 09:20 AM
Freakin Nora. It works great. Much faster!

Sweet! :D

Jonnyfive
Mar 1, 2007, 09:23 AM
I got an error in terminal when trying to enter the last line... am I doing something wrong?

sqlite> sqlite> vacuum subjects;
SQL error: near "sqlite": syntax error

redwin11
Mar 1, 2007, 09:31 AM
I got an error in terminal when trying to enter the last line... am I doing something wrong?

Yes

Jonnyfive
Mar 1, 2007, 09:35 AM
Yes

Well what am I doing wrong?

wrldwzrd89
Mar 1, 2007, 09:36 AM
Hmm... I'll have to try this once I return to my Mac. My Mail.app hasn't been particularly sluggish.

iMeowbot
Mar 1, 2007, 09:44 AM
Well what am I doing wrong?

You don't want to type the sqlite> part, that's the command prompt for the program.

redwin11
Mar 1, 2007, 09:45 AM
Well what am I doing wrong?

Copy and paste it, ensures there's no errors for a start.

Jonnyfive
Mar 1, 2007, 09:47 AM
You don't want to type the sqlite> part, that's the command prompt for the program.

Copy and paste it, ensures there's no errors for a start.

Well, I guess that means copy and paste wouldn't work... Smart Guy.

wrldwzrd89
Mar 1, 2007, 09:49 AM
Well, I guess that means copy and paste wouldn't work... Smart Guy.
It does work, you just have to make sure you don't accidentally copy the prompt. Also, don't try to copy both lines at once, as it won't work.

Jonnyfive
Mar 1, 2007, 09:51 AM
It does work, you just have to make sure you don't accidentally copy the prompt. Also, don't try to copy both lines at once, as it won't work.

Yeah I got it to work. Just saying that copying and pasting everything written on the page isn't the solution.

redwin11
Mar 1, 2007, 09:53 AM
Well, I guess that means copy and paste wouldn't work... Smart Guy.

You're the one who's struggling with three lines of text!

Jonnyfive
Mar 1, 2007, 09:55 AM
You're the one who's struggling with three lines of text!

Touche.

MacBoobsPro
Mar 1, 2007, 09:59 AM
I copied and pasted all 3 lines and it seems to have worked for me? :confused:

So for people who are struggling can someone post a step by step? We are here to help people right? Maybe I didnt do it right but it seems faster to me.

mainstreetmark
Mar 1, 2007, 10:32 AM
I copied and pasted all 3 lines and it seems to have worked for me? :confused:

So for people who are struggling can someone post a step by step? We are here to help people right? Maybe I didnt do it right but it seems faster to me.

Really?!?

Step 1: Quit mail
Step 2: Open Terminal (use Spotlight to find 'Terminal')
Step 3: Copy this bit and paste it in the terminal directly:

cd ~/Library/Mail
sqlite3 Envelope\ Index
vacuum subjects;

... and press return.
Step 4: When it says "sqlite>" again, close Terminal
Step 5: Open Mail

If you still have trouble, it's time to reconsider your ability to own a computer.

Mine sped up significantly!

devilot
Mar 1, 2007, 10:35 AM
Really?!?Thanks, it's appreciated, I'm sure by others too timid to now reply. :rolleyes:

Anyway, definitely speeds things up. Can't wait to get home and try this out on my iMac. That fool needs the speed up more than my MB. :o

mainstreetmark
Mar 1, 2007, 10:43 AM
I'll do you all one better. Here's a quick app to do it for you:

A buddy of mine pointed out that you can use iCal, set a monthly alarm, and for the alarm action, use 'Run Script' and point to this file, and set it to go off every month. Neat, eh?

IJ Reilly
Mar 1, 2007, 10:49 AM
I just tried the easy way, deleting the Envelope file. After about five minutes of rebuilding, it had been reduced from 10 Mb to about 8 Mb. Speed improvements? Negligible, at best. Maybe mine wasn't large enough to make much difference, but I am unimpressed.

mainstreetmark
Mar 1, 2007, 10:51 AM
Mine went from 11 to 5. It used to be I could see the inbox populating up with messages. Would take seconds sometimes. Now, when Mail stars, my inbox just appears with no delay.

MacBoobsPro
Mar 1, 2007, 10:52 AM
Thankyou Mainstreetmark.

Apparently I did do it right first time but your script will help people who are put off by using the Terminal.

:)

IJ Reilly
Mar 1, 2007, 11:02 AM
Mine went from 11 to 5. It used to be I could see the inbox populating up with messages. Would take seconds sometimes. Now, when Mail stars, my inbox just appears with no delay.

Yours started out about the same size, but decreased by over 50% and mine by 20%. Maybe that's why you saw a significant performance boost and I did not. Any explanation?

In the interests of science I tried running the terminal commands. No change in file size.

devilot
Mar 1, 2007, 11:03 AM
Any explanation?I believe the different articles point the blame at IMAP vs. POP accounts. :o

Note, my file size also didn't really change, but the app still loads the inbox super speedy style like MSM mentioned.

IJ Reilly
Mar 1, 2007, 11:12 AM
I believe the different articles point the blame at IMAP vs. POP accounts. :o

Yes, but based only on "anecdotal" evidence. I have both POP and IMAP accounts.

SpaceMagic
Mar 1, 2007, 11:13 AM
It really works!

shadowfax
Mar 1, 2007, 11:29 AM
I believe the different articles point the blame at IMAP vs. POP accounts. :oTheir anecdotal evidence is very weak, honestly. if you read the thread on their site, you'll see that half the respondents contradict their POP hypothesis. It doesn't even seem to be based on the size of the database: I have 1 POP and 1 IMAP account, and my envelope was only like 3.5 MB. It got cranked down to 2.5 MB, and it's made a pretty noticeable difference.

There must be some kinda funny bug that gets introduced into the database. I guess if someone were into MySQL they could have a look at the database before and after... I am not up for that.

Analog Kid
Mar 1, 2007, 01:55 PM
What is it deleting exactly?

What are the consequences?
I had the same question-- it really is an optimization, you aren't losing any information:
http://www.sqlite.org/lang_vacuum.html

What I don't know is how this is better than just doing a "rebuild mailbox" from the menus...

mainstreetmark
Mar 1, 2007, 02:02 PM
I had the same question-- it really is an optimization, you aren't losing any information:
http://www.sqlite.org/lang_vacuum.html

What I don't know is how this is better than just doing a "rebuild mailbox" from the menus...

so why not just set the auto vacuum pragma (http://www.sqlite.org/pragma.html#pragma_auto_vacuum) ?

Analog Kid
Mar 1, 2007, 02:08 PM
so why not just set the auto vacuum pragma (http://www.sqlite.org/pragma.html#pragma_auto_vacuum) ?
"It is only possible to modify the value of the auto-vacuum flag before any tables have been created in the database. No error message is returned if an attempt to modify the auto-vacuum flag is made after one or more tables have been created."

I think it would have to be set by the application before it builds the database...

Poff
Mar 1, 2007, 02:14 PM
I get these two error messages:

-bash: sqlite3: command not found
-bash: vacuum: command not found

does this have anything to do with me having 10.3.9?

nagromme
Mar 1, 2007, 02:26 PM
What I don't know is how this is better than just doing a "rebuild mailbox" from the menus...

I'm curious too. I'm going to try a GUI method (I'm not in a hurry so no need for Terminal). Either deleting the index, or "Rebuild." (The latter being harder because you must choose each folder individually.) Does Rebuild do the same thing, or something different?

mainstreetmark
Mar 1, 2007, 03:12 PM
I get these two error messages:

-bash: sqlite3: command not found
-bash: vacuum: command not found

does this have anything to do with me having 10.3.9?

No. Read up a few posts. I clarified, and even wrote a little script for you to click on.

Poff
Mar 1, 2007, 04:05 PM
No. Read up a few posts. I clarified, and even wrote a little script for you to click on.

Hmm.. I did the exact same thing you wrote in your post, and got those error messages.

Even after I used your script, I got an error message.

Last login: Thu Mar 1 23:02:22 on ttyp1
Welcome to Darwin!
Gro-og-Erlend-Dietrich-Handelands-maskin:~ groogerlend$ /Users/groogerlend/Desktop/speedup_mail.sh; exit
Optimizing....
/Users/groogerlend/Desktop/speedup_mail.sh: line 7: sqlite3: command not found
logout
[Prosess utført]


Edit: so is the file supposed to be edited named "Envelope"? And it's supposed to be in library/mail? Because I don't have that file there, so I guess that's the reason it won't work. My guess is still that it has to do with 10.3.9 and mail.app 1.3.

But a file named "vacuum" was created with the script, tho. :)

angelwatt
Mar 1, 2007, 04:24 PM
Well my envelope index is only 896KB so doubt this will do anything for me. My mail already feels speedy enough.

mkrishnan
Mar 1, 2007, 04:43 PM
Hmm.. I did the exact same thing you wrote in your post, and got those error messages.

I did a little Googling, and I'm not at all sure that sqlite is in Panther. But then I also know that Mail became drastically slower for me (with a huge amount of mail storage... about 1.6 gigs, and 7-8 mail accounts), particularly after some use, in Tiger than it had been in Panther. So... maybe those two things are not completely coincidental? :o

dukeblue91
Mar 1, 2007, 04:45 PM
Sweet :D
I wondered why mail was getting so slow, but this worked like a charm with a huge speed boost.

I wonder though, how often should this be done?

wrldwzrd89
Mar 1, 2007, 04:49 PM
Whoa, did that ever speed things up. :D

nagromme
Mar 1, 2007, 04:56 PM
Does Rebuild do the same thing, or something different?

FWIW: I trued Rebuild on my main mailboxes, with the following result:

1. Envelope Index was 14MB, now still 14MB.

2. New (tiny) file created: Envelope Index-journal

3. A few dozen messages spanning 3 years and multiple accounts have been changed to Unread and Unreplied, even though I did read and reply to them. (Not a huge deal, but any email weirdness is disturbing when your POP records are important.)

4. I don't notice a speed difference--but then Mail never seemed all that slow to me anyway. (It should, I have zillions of messages going back a decade +.)

So... Rebuild has SOMETHING to do with Envelope Index, but it didn't seem to optimize anything.

Mitthrawnuruodo
Mar 1, 2007, 05:07 PM
I didn't get much speedup from this (and I forgot to check the size of the Envelope Index before I did it, now it's 2.8 MB), but that might be because I just today did the ultimate Mail.app optimizer: I cleared out my Inbox and Sent mail boxes. Going from several hundred sent mail and 50-100 messages in the inbox, to just a few (current ones) in each folder, makes Mail unbelievable more snappy.

Raid
Mar 1, 2007, 05:09 PM
I'll do you all one better. Here's a quick app to do it for you:

A buddy of mine pointed out that you can use iCal, set a monthly alarm, and for the alarm action, use 'Run Script' and point to this file, and set it to go off every month. Neat, eh? Very cool, thanks! I was wondering since I have Macaroni (http://www.atomicbird.com/macaroni/index.html) on my machine could I add this as a "New Job"? I've never tried adding to the program before, but I think it's possible (and not as messy as the iCal alarm action IMHO). Any thoughts?

Analog Kid
Mar 1, 2007, 06:01 PM
So... Rebuild has SOMETHING to do with Envelope Index, but it didn't seem to optimize anything.

Hey, thanks for trying this. Guess I'll give the vacuum a shot, not that I've ever had much of a problem with speed in Mail.

I wonder if you and I aren't seeing a problem because we each have "zilliions" of messages-- that is we never delete anything. The documentation on the vacuum command seems to imply fixes fragmentation due to deletions and insertions.

SimonTheSoundMa
Mar 1, 2007, 06:21 PM
Cool. Added to the OS X/BSD periodic weekly maintenance.

termite
Mar 1, 2007, 06:52 PM
A buddy of mine pointed out that you can use iCal, set a monthly alarm, and for the alarm action, use 'Run Script' and point to this file, and set it to go off every month. Neat, eh?Running your script while Mail.app is still running strikes me as a very dangerous idea. Any idea whether or not Mail.app will use locking to prevent this from toasting your db? Or does your script check to see if Mail.app is running and bail out if it is?

mkrishnan
Mar 1, 2007, 07:17 PM
One could stop Mail either using shell scripting or Applescripting if one really wanted to do so, yeah. (Also P.S. I just looked it up... SQLite is designed to fail if the vacuum command is given while a transaction is active, so it shouldn't do anything catastrophic, but...)

Anyway, I just did mine. Mein Gott im Himmel! It's amazing! :D

Apple really needs to incorporate this somehow into its own maintenance routines.

Analog Kid
Mar 1, 2007, 07:21 PM
I just ran "vacuum;" to optimize all tables. File size went from 71-67MB, and I'm not seeing much of a difference. It was fast enough for me before, I guess. Followed that up with a "vacuum subjects;" in case I'd misread the directive and no change to anything.

Searching seems a little faster, but that could just be me looking for something to improve.

mainstreetmark
Mar 2, 2007, 10:24 AM
Running your script while Mail.app is still running strikes me as a very dangerous idea. Any idea whether or not Mail.app will use locking to prevent this from toasting your db? Or does your script check to see if Mail.app is running and bail out if it is?

Na, my "highly complex" script quits mail (via applscript), runs the sqlite thing, and opens it back up (assuming it's in /Applicaitons). Should be safe.

And, yeah, this whole process requires sqlite3, which may not be installed as an independent tool in Panther. sqlite is a stand-alone database that many people include in their apps, and sqlite3 is just the client which connects to the 'server'.

nateDEEZY
Mar 2, 2007, 01:47 PM
I definitely have to try this when I get home, thanks!

dvader
Mar 2, 2007, 02:22 PM
I get nervous when I read that it deletes things in order to make it go faster. Exactly what gets deleted and can it effect other applications?

smueboy
Mar 2, 2007, 02:30 PM
My Envelope index went from 3.7 to 3.4 MB (i'm using 3 IMAP accounts). But Mail is now super fast, so i'm happy.

:)

nagromme
Mar 2, 2007, 03:03 PM
I'd say then that it's not the file size shrinkage that is key--but the correction of errors in the file.

smueboy
Mar 2, 2007, 05:01 PM
It "eliminates free pages, aligns table data to be contiguous, and otherwise cleans up the database file structure"