Can you download a list of URLs using automator?

Discussion in 'Mac Programming' started by whitedragon101, Oct 30, 2014.

  1. whitedragon101 macrumors 65816

    Joined:
    Sep 11, 2008
    #1
    I use a website that runs a service I will shortly loose. It lists all my cricket clubs member details just changing member number for each URL. They don't want to make it easy for you to download a csv of all the members so you keep using them. I have made a webpage (on my own server) that lists every member id as a link.

    www.sportsclub-manager.com/member13424.asp
    www.sportsclub-manager.com/member45447.asp
    etc
    (1500 links)

    I want to download each page as a full webarchive.

    I have tried downloadthemall and scrapbook firefox add ons but there is a table that appears 1second after the page loads that doesn't get downloaded. If I save manually as a webarchive it does.

    Can automator do something this i.e

    load list of URLs from webpage
    start loop
    load link x to safari
    wait 3 seconds
    save content to file
    end loop
     
  2. The Doctor11 macrumors 603

    The Doctor11

    Joined:
    Dec 15, 2013
    Location:
    New York
    #2
    You made two threads :eek:
     

    Attached Files:

  3. whitedragon101 thread starter macrumors 65816

    Joined:
    Sep 11, 2008
    #3
    I can't delete the other one but I have deleted the content and pointed it here.
     
  4. whitedragon101 thread starter macrumors 65816

    Joined:
    Sep 11, 2008
    #5
    Thank you very much. I nearly have it. I'd really appreciate it if you could have a look at this and edit it so the the variable theURL is entered as the filename to save (by default it uses the page title which is the same for every page).

    Code:
    tell application "Safari"
    	activate
    	repeat with theURL in theList
    		set the URL of front document to theURL
    		tell application "System Events"
    			tell process "Safari"
    				tell menu 1 of menu bar item "File" of menu bar 1
    					repeat while enabled of menu item "Save as…" is false
    						delay 0.1
    					end repeat
    				end tell
    				keystroke "s" using command down -- Save As…
    				repeat until sheet 1 of window 1 exists
    					delay 0.1
    				end repeat
    				keystroke "d" using command down -- save to Desktop
    				tell sheet 1 of window 1
    					click pop up button 1 of group 1
    					keystroke return -- Web Archive
    					keystroke return -- Save
    					if sheet 1 exists then -- that is "sheet 1 of sheet 1"
    						click button "Replace" of sheet 1
    					end if
    				end tell
    			end tell
    		end tell
    	end repeat
    end tell
     
  5. kryten2 macrumors 6502a

    Joined:
    Mar 17, 2012
    Location:
    Belgium
    #6
    Try this :

    Code:
    tell sheet 1 of window 1
    set value of text field 1 to theURL -- theURL as the filename to save
    click pop up button 1 of group 1
    keystroke return -- Web Archive
    keystroke return -- Save
    if sheet 1 exists then -- that is "sheet 1 of sheet 1"
    click button "Replace" of sheet 1
    end if
    end tell
    
     
  6. whitedragon101 thread starter macrumors 65816

    Joined:
    Sep 11, 2008
    #7
    Awesome thanks. It worked perfectly :)
     

Share This Page