Large Mailer Script - Scalability?

Discussion in 'Web Design and Development' started by Aperture, Jul 31, 2009.

  1. Aperture macrumors 68000

    Aperture

    Joined:
    Mar 19, 2006
    Location:
    PA
    #1
    I'm going to be designing a script to check a MySQL database nightly (via a cronjob) to find users who meet certain criteria. It will then need to notify that person via a short email.

    This is going to be a fairly large system and it will need to probably email around ~1000ish people an night. (Disclaimer: NO, this is not spam and is on an opt-in basis)

    Have any of you accomplished a mailer script on a large scale? What do I need to focus on to ensure scalability and reliability?

    Oh, and I'm planning to write this in PHP.
     
  2. sl1200mk2 macrumors 6502

    Joined:
    Oct 17, 2006
    #2
    I work in the hosting industry, so hopefully my advice is helpful. In my mind, it's not really the script or scripting per se that's the issue, it's the MTA (mail server) the script is injecting to that's your real concern. This is where the bulk of your reliability and scalability will come from.

    If you're running this on a LAMP stack the MTA is likely Sendmail. Sendmail (most modern MTAs for that matter) can easily accommodate 1k emails, however, depending on the environment (shared hosting, VPS, dedicated server, etc) and the resources available you may want to throttle the number of messages introduced to the mail server in batches. As a hosting company, we typically like to see 250 messages ~ 30-60 seconds. Tossing 1,000 messages at once into an already busy mail server can introduce quite a bit of load and impact other users. If this is a VPS or dedicated server it shouldn't be a problem at all, but I would still advise some level of throttling.

    If you follow that model and keep the list 'clean' you shouldn't have too many problems. By clean I mean regularly checking / processing bounce messages and removing the dead and/or non-functioning email addresses. Anything less than 10,000 messages really isn't anything your average mail server will have a hard time handling, but again, your environment and likely hosting company will have guidance and limits on the amounts of SMTP traffic you'll be allowed to process in a given period. Seek their advice.

    Good luck -
     
  3. Aperture thread starter macrumors 68000

    Aperture

    Joined:
    Mar 19, 2006
    Location:
    PA
    #3
    Thanks a bunch, that really helps. This is going to be ran on a dedicated web server, and the mail server on it is not in use... So I don't have to worry about it impacting other users. Thanks again.
     

Share This Page