Im getting pwned by 10GB PDF files! 8GB RAM is not enough.. over 5.3GB in SWAP!

Discussion in 'MacBook Pro' started by Vyruz Reaper, Aug 7, 2010.

  1. Vyruz Reaper macrumors regular

    Joined:
    Dec 23, 2009
    #1
    Thanks to everyone helping me out on my other thread. This is geting ridiculous though. All I need to do is take 10GB PDFs and do some research from them, then break them up into smaller pdf files and organize them. I tried taking a screenshot when my swap was over 5GB but my system was waay to slow and unresponsive. It took about 15 minutes before I regained control of my MBP.

    I have Kingston 8GB Ram, 2.4 i5, Hi Res screen, 5400RPM HD.

    here is the last screenshot I could pull off before it locked up for 15 minutes..

    should I return my Kingston RAM and try buying Crucial? Or is my problem REALLY a SSD... I asked around and no one knows **** with the research team. And the Prof. I do it for uses Windows and he doesnt know anything either. lol

    [​IMG]
     
  2. Frosties macrumors 6502a

    Frosties

    Joined:
    Jun 12, 2009
    Location:
    Sweden
    #2
    Can you not make the pdf files to text files instead. Pdf is not fun when you run into these sizes.
     
  3. vant macrumors 65816

    Joined:
    Jul 1, 2009
    #3
    8GB of RAM is the limitation here. Getting a SSD would improve speeds as you are now using the HD for normal RAM functions.
     
  4. Vyruz Reaper thread starter macrumors regular

    Joined:
    Dec 23, 2009
    #4
    The PDF files are actually pictures taken of Books from the 1920s. Then I complied all of them into a PDF file. Unless there is a better method I dont know of this is only what i got.

    Here is another Screenshot

    [​IMG]
     
  5. vant macrumors 65816

    Joined:
    Jul 1, 2009
  6. Vyruz Reaper thread starter macrumors regular

    Joined:
    Dec 23, 2009
    #6
    If it was a 50-100 page book I would. The books are about 3000 pages each, thats too many files to keep separate.
     
  7. Hisdem macrumors 6502a

    Hisdem

    Joined:
    Jul 3, 2010
    Location:
    Boca Raton, FL
    #7
    You are having A LOT of page outs/swap etc. Really, you should try finding another way to keep these books, because the system is being stressed. Since 8GB is as far as you go on a MBP, I can't see much to be done hardware wise besides a SSD. :(

    Oh and BTW, you're using Chrome right? I use Safari and have noticed that over some time open it can take up a good 300MB of RAM. Not sure if this happens with other browsers though.

    I don't know much about this kind of thing but I assume building a slideshow out of those images would result in even worse performance right?
     
  8. Vyruz Reaper thread starter macrumors regular

    Joined:
    Dec 23, 2009
    #8
    So is it bad for my computer to continue what I am doing? I am over 1 million for both page in and page outs now. I just ran OnyX and my swap is down to 250mb. I think I can take the pictures and just separate them while in jpeg format before converting them to pdf. WAY more time consuming but I can cut the size of pdf files to under 1GB each and I will end up with around 40-50 PDFs
     
  9. Hisdem macrumors 6502a

    Hisdem

    Joined:
    Jul 3, 2010
    Location:
    Boca Raton, FL
    #9
    I'm not really sure if it could harm your computer, since it's RAM and not CPU that's being stressed. However, as it's being used way over the capacity, it could result on something. But it will be better for you if you wait for someone with more knowledge than me to clear this up ;)
     
  10. tdgrn macrumors 6502

    Joined:
    May 1, 2008
    Location:
    Little Rock, AR
    #10
    Have you tried using Adobe Reader? Normally I wouldn't suggest it, however by looking at your iStat, it shows that you have 2.38 GBs of RAM that isn't Active...

    Maybe Preview isn't able to access more than 3.75 GBs since it is a single app. Seems like I remember something about Photoshop CS4 couldn't access more than 3.75 GB's of RAM. Another thing to check is to make sure that Preview is not opening in 32bit mode. The other thing that I would recommend before buying an SSD is to boot your Mac into a 64bit Kernel. As soon as you boot your computer up, press and hold your 6 and 4 keys together... or download and install this app - K64enabler -> http://www.macupdate.com/info.php/id/32300/k64enabler
     
  11. tshort macrumors regular

    Joined:
    Jul 20, 2007
    #11
    Do they need to be in color?

    Have you considered scanning and saving as B&W? It will significantly reduce the size of the files.
     
  12. Vyruz Reaper thread starter macrumors regular

    Joined:
    Dec 23, 2009
    #12
    Next set of pictures I take will be in Black and white for sure
     
  13. theLimit macrumors 6502a

    theLimit

    Joined:
    Jan 30, 2007
    Location:
    up tha holler, acrost tha crick
    #13
    The slowdown/lockup problem is because the PDF file is bigger than your total RAM. Making the files smaller, say 500 pages each insted of 3000, would add more steps to your workflow but allow you to work more smoothly without the slowdowns. The only component you're stressing right now is the hard drive, as it is constantly being forced to read and write from all the paging. An SSD would speed things up a bit, as it is faster than an HDD. But it wouldn't make a huge difference because RAM is still exponentially faster. You would also be stressing it just as much as your hard drive currently. Since more RAM isn't an option on your machine, I would split the files to work within the RAM limitation.

    The problem is with the amount of RAM, changing brands won't matter.
     
  14. Vyruz Reaper thread starter macrumors regular

    Joined:
    Dec 23, 2009
    #14
    Thanks


    So I rebooted into 64bit mode and now the lockup times are shorter so it did help. I still do get lockup however....


    SOOOOO the million dollar question is..

    How to I Break my current 10GB PDF into 50 smaller PDF files? I tried selecting 100 or so pages then printing it as anther pdf? But then I get another lockup longer than other lockups and I have to wait a long time. Is there a better way rather than just printing it as a pdf?


    thanks!
     
  15. sth macrumors 6502a

    Joined:
    Aug 9, 2006
    Location:
    The old world
    #15
    You could:
    - Change your workflow (e.g. multiple files, lower resolution)
    - Get an SSD drive (will speed up the swapping a bit but will still be slow)
    - Get a Mac Pro. Seriously, extremes like this is what they are made for.

    As for splitting PDFs:
    I would suggest command-line tools like pdftk (you can install it via MacPorts or find some prebuilt versions via Google if you don't have MacPorts installed). They will probably work faster than graphical tools in this case since they don't need to load the whole document for displaying.
     
  16. Corndog5595 macrumors 65816

    Joined:
    Jul 16, 2010
    #16
    You could extract the text from the PDF files in automator. That should reduce them to about 10% original size.

    There might be a better solution though.
     
  17. phairphan macrumors 6502a

    phairphan

    Joined:
    Sep 21, 2005
    Location:
    Reject Beach
    #17
    I doubt these have been OCR'd and, even they had, converting these sorts of formatted documents to text almost always gives you garbage. The best thing to do for now is to break the behemoth document into more manageable parts.
     
  18. jaykk macrumors 6502a

    Joined:
    Jan 5, 2002
    Location:
    CA
    #18
    Automater is very useful for these situations, somehow I didn't find any default action for splitting by range (you can extract individual, or merge 2 pdf etc.) You can try this utility

    Moonwalk - http://www.jamapi.com/pr/mw/
     
  19. plasticphyte macrumors 6502

    Joined:
    Nov 9, 2009
    #19
    If you're not wanting to/can't change your work flow, then you'll need to increase the amount of RAM in your system to 12GB, or even 16GB.

    Have a read through these articles which may help your reduce file sizes of current/future PDF's - you might want to make two copies, one being the large original high quality one, the other being one of a lower file size & lesser quality so that you can work quickly from/with them.

    http://www.planetpdf.com/enterprise/article.asp?ContentID=6112

    http://www.google.com.au/url?sa=t&s...sg=AFQjCNHrn4i2bPtuUYPTxWXH6rXdEn6Xug&cad=rja

    http://blog.nitropdf.com/index.php/2008/02/11/5-tricks-to-shrinkreduce-pdf-file-size/
     
  20. J&JPolangin macrumors 68030

    Joined:
    Jul 5, 2008
    Location:
    Thule GL @ the TOW
    #20
    ...its a MBP, 8Gb is the max...
     
  21. revelated macrumors 6502a

    revelated

    Joined:
    Jun 30, 2010
    #21
    I think the SSD is the only answer for a native solution.

    The optimal thing to do would be to reprocess the PDF properly. 3,000 pages @ 10GB tells me that it's not an optimized PDF as you should be around the 2GB mark or thereabouts for that many pages, assuming a properly optimized PDF. (I work in document imaging). Even with images included you'd still not exceed 5GB at the outside per PDF.

    If the PDF were space usage audited you'd probably see all of the size as image data - which means it wasn't properly converted for reading purposes - and some embedded flotsam.
     
  22. The Beatles macrumors regular

    Joined:
    Jun 16, 2010
    #22
    You have ONE file thats ten gigs? If you try to push around a file that size, its going to be slow on any computer. I'd look at your workflow rather than your hardware. Especially when it comes to SSDs. A faster hard drive isnt going to be the solution. Everybody is SSD happy now. Unless money is no object there are better ways for improving performance. And as i said, in your case, its a workflow issue. You have one container (PDF file) holding all your info. Thats just bloated. Put them in folders as individual files. Categorize them, its your best solution. If you have Adobe Bridge, you can use that to organize them as well.
     
  23. jjahshik32 macrumors 603

    Joined:
    Sep 4, 2006
    #23
    Looks like somebody needs to buy a new up coming Mac Pro. :cool:
     
  24. Vyruz Reaper thread starter macrumors regular

    Joined:
    Dec 23, 2009
    #24
    haha yeah right... not with the budget the UC system is putting us on :D
     
  25. Koifim macrumors newbie

    Joined:
    May 20, 2009
    Location:
    The Netherlands
    #25
    Try purging you RAM, in the terminal type 'purge'. I do this when I have used VM software like Parallels. When I quit that application, my free RAM will not increase. It doesn't hurt anything, your system will just 'hang' for a few seconds, after that it's all back to normal.
     

Share This Page