Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Im getting pwned by 10GB PDF files! 8GB RAM is not enough.. over 5.3GB in SWAP!

V

Vyruz Reaper

macrumors regular
Original poster
Dec 23, 2009
134
0
Thanks to everyone helping me out on my other thread. This is geting ridiculous though. All I need to do is take 10GB PDFs and do some research from them, then break them up into smaller pdf files and organize them. I tried taking a screenshot when my swap was over 5GB but my system was waay to slow and unresponsive. It took about 15 minutes before I regained control of my MBP.

I have Kingston 8GB Ram, 2.4 i5, Hi Res screen, 5400RPM HD.

here is the last screenshot I could pull off before it locked up for 15 minutes..

should I return my Kingston RAM and try buying Crucial? Or is my problem REALLY a SSD... I asked around and no one knows **** with the research team. And the Prof. I do it for uses Windows and he doesnt know anything either. lol

screenshot20100807at356.png
 
Can you not make the pdf files to text files instead. Pdf is not fun when you run into these sizes.
 
8GB of RAM is the limitation here. Getting a SSD would improve speeds as you are now using the HD for normal RAM functions.
 
The PDF files are actually pictures taken of Books from the 1920s. Then I complied all of them into a PDF file. Unless there is a better method I dont know of this is only what i got.

Here is another Screenshot

screenshot20100807at416.png
 
You are having A LOT of page outs/swap etc. Really, you should try finding another way to keep these books, because the system is being stressed. Since 8GB is as far as you go on a MBP, I can't see much to be done hardware wise besides a SSD. :(

Oh and BTW, you're using Chrome right? I use Safari and have noticed that over some time open it can take up a good 300MB of RAM. Not sure if this happens with other browsers though.

I don't know much about this kind of thing but I assume building a slideshow out of those images would result in even worse performance right?
 
Hisdem said:
You are having A LOT of page outs/swap etc. Really, you should try finding another way to keep these books, because the system is being stressed. Since 8GB is as far as you go on a MBP, I can't see much to be done hardware wise besides a SSD. :(
Click to expand...

So is it bad for my computer to continue what I am doing? I am over 1 million for both page in and page outs now. I just ran OnyX and my swap is down to 250mb. I think I can take the pictures and just separate them while in jpeg format before converting them to pdf. WAY more time consuming but I can cut the size of pdf files to under 1GB each and I will end up with around 40-50 PDFs
 
I'm not really sure if it could harm your computer, since it's RAM and not CPU that's being stressed. However, as it's being used way over the capacity, it could result on something. But it will be better for you if you wait for someone with more knowledge than me to clear this up ;)
 
Have you tried using Adobe Reader? Normally I wouldn't suggest it, however by looking at your iStat, it shows that you have 2.38 GBs of RAM that isn't Active...

Maybe Preview isn't able to access more than 3.75 GBs since it is a single app. Seems like I remember something about Photoshop CS4 couldn't access more than 3.75 GB's of RAM. Another thing to check is to make sure that Preview is not opening in 32bit mode. The other thing that I would recommend before buying an SSD is to boot your Mac into a 64bit Kernel. As soon as you boot your computer up, press and hold your 6 and 4 keys together... or download and install this app - K64enabler -> http://www.macupdate.com/info.php/id/32300/k64enabler
 
Do they need to be in color?

Have you considered scanning and saving as B&W? It will significantly reduce the size of the files.
 
The slowdown/lockup problem is because the PDF file is bigger than your total RAM. Making the files smaller, say 500 pages each insted of 3000, would add more steps to your workflow but allow you to work more smoothly without the slowdowns. The only component you're stressing right now is the hard drive, as it is constantly being forced to read and write from all the paging. An SSD would speed things up a bit, as it is faster than an HDD. But it wouldn't make a huge difference because RAM is still exponentially faster. You would also be stressing it just as much as your hard drive currently. Since more RAM isn't an option on your machine, I would split the files to work within the RAM limitation.

The problem is with the amount of RAM, changing brands won't matter.
 
theLimit said:
The slowdown/lockup problem is because the PDF file is bigger than your total RAM. Making the files smaller, say 500 pages each insted of 3000, would add more steps to your workflow but allow you to work more smoothly without the slowdowns. The only component you're stressing right now is the hard drive, as it is constantly being forced to read and write from all the paging. An SSD would speed things up a bit, as it is faster than an HDD. But it wouldn't make a huge difference because RAM is still exponentially faster. You would also be stressing it just as much as your hard drive currently. Since more RAM isn't an option on your machine, I would split the files to work within the RAM limitation.

The problem is with the amount of RAM, changing brands won't matter.
Click to expand...

Thanks


So I rebooted into 64bit mode and now the lockup times are shorter so it did help. I still do get lockup however....


SOOOOO the million dollar question is..

How to I Break my current 10GB PDF into 50 smaller PDF files? I tried selecting 100 or so pages then printing it as anther pdf? But then I get another lockup longer than other lockups and I have to wait a long time. Is there a better way rather than just printing it as a pdf?


thanks!
 
You could:
- Change your workflow (e.g. multiple files, lower resolution)
- Get an SSD drive (will speed up the swapping a bit but will still be slow)
- Get a Mac Pro. Seriously, extremes like this is what they are made for.

As for splitting PDFs:
I would suggest command-line tools like pdftk (you can install it via MacPorts or find some prebuilt versions via Google if you don't have MacPorts installed). They will probably work faster than graphical tools in this case since they don't need to load the whole document for displaying.
 
You could extract the text from the PDF files in automator. That should reduce them to about 10% original size.

There might be a better solution though.
 
I doubt these have been OCR'd and, even they had, converting these sorts of formatted documents to text almost always gives you garbage. The best thing to do for now is to break the behemoth document into more manageable parts.
 
If you're not wanting to/can't change your work flow, then you'll need to increase the amount of RAM in your system to 12GB, or even 16GB.

Have a read through these articles which may help your reduce file sizes of current/future PDF's - you might want to make two copies, one being the large original high quality one, the other being one of a lower file size & lesser quality so that you can work quickly from/with them.

http://www.planetpdf.com/enterprise/article.asp?ContentID=6112

http://www.google.com.au/url?sa=t&s...sg=AFQjCNHrn4i2bPtuUYPTxWXH6rXdEn6Xug&cad=rja

http://blog.nitropdf.com/index.php/2008/02/11/5-tricks-to-shrinkreduce-pdf-file-size/
 
I think the SSD is the only answer for a native solution.

The optimal thing to do would be to reprocess the PDF properly. 3,000 pages @ 10GB tells me that it's not an optimized PDF as you should be around the 2GB mark or thereabouts for that many pages, assuming a properly optimized PDF. (I work in document imaging). Even with images included you'd still not exceed 5GB at the outside per PDF.

If the PDF were space usage audited you'd probably see all of the size as image data - which means it wasn't properly converted for reading purposes - and some embedded flotsam.
 
Vyruz Reaper said:
So is it bad for my computer to continue what I am doing? I am over 1 million for both page in and page outs now. I just ran OnyX and my swap is down to 250mb. I think I can take the pictures and just separate them while in jpeg format before converting them to pdf. WAY more time consuming but I can cut the size of pdf files to under 1GB each and I will end up with around 40-50 PDFs
Click to expand...

You have ONE file thats ten gigs? If you try to push around a file that size, its going to be slow on any computer. I'd look at your workflow rather than your hardware. Especially when it comes to SSDs. A faster hard drive isnt going to be the solution. Everybody is SSD happy now. Unless money is no object there are better ways for improving performance. And as i said, in your case, its a workflow issue. You have one container (PDF file) holding all your info. Thats just bloated. Put them in folders as individual files. Categorize them, its your best solution. If you have Adobe Bridge, you can use that to organize them as well.
 
Try purging you RAM, in the terminal type 'purge'. I do this when I have used VM software like Parallels. When I quit that application, my free RAM will not increase. It doesn't hurt anything, your system will just 'hang' for a few seconds, after that it's all back to normal.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back