Become a MacRumors Supporter for $25/year with no ads, private forums, and more!

How can I download hundreds of PDFs at once?

Silly John Fatty

macrumors 6502a
Original poster
Nov 6, 2012
978
30
Hi community,

so right now, being a member of my local university, I have access to scientific literature online, and basically there's hundreds of PDFs and eBooks I can download for free. However, my membership expires soon and I won't have access to all these great works anymore.

There are literally hundreds, I believe thousands of PDFs and ePub files I want to download. I've downloaded around 300 already, but it's tiring. So I was wondering if there was a way to make a script or something, to download them automatically.

Once I've chosen a book from the search results, the webpage will look like this here:

Website Screenshot.png

I have to click on those two links to download the PDF and the ePub file each time, and like I said, it's quite tiring after a while. This is the process I'd like to automate. So basically I would open hundreds of tabs with hundreds of different books, and then I'd like to hit a button and the thing just downloads it all for me.

Is this even possible? How complicated is this?

Thanks people! ;)
 

gilby101

macrumors 6502
Mar 17, 2010
383
154
Tasmania
Is this even possible? How complicated is this?
Possible: Yes, if your fingers can do it, so can your computer.
Complicated: Probably, also yes! You need to be a scripting wizard to do it cleanly. But, there are utilities that will download a whole web site.

Of course, you will probably be breaking your institution's licensing agreement with Springer. So your question may be out of order on this web site.
 
Comment

funkypepper

macrumors member
Feb 6, 2007
95
5
You can use `curl` or `wget` command line utilities to download these files, but you need to write a script.

Basically you need to analyse the URL of the downloads, and pass an argument for the changing parts. This may not be possible if the site is using random URLs.

I believe a Google search with the relevant keywords will give you what you need.
 
Comment
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.