Applescript: determine RGB value at specific pixel on screen

Discussion in 'Mac Programming' started by sammich, Mar 2, 2010.

  1. sammich macrumors 601


    Sep 26, 2006
    Just wondering if there was some way to get the RGB value for a specific point on the screen using Applescript. I'm doing some (flash) game automating and sampling the color means I can check if the correct dialog has appeared before clicking.

    I've googled around and found a couple of close hits, but no solutions:

    - This post has the same idea as above, from 2006 and no solution
    - This will get an RGB value from an image.

    As the first link, I'd like to do:
    if coloratpixel(x, y) is equal to {r, g, b} then
          click(x, y)
    If there exists no solution, could someone point me in a direction where I can take selection screen shots (like command-shift-4) with AS or from the shell and I can use the 2nd link above to grab the RGB value. I know I can take full screen screenshots, but that would be unwieldy if I were to sample the screen several times a second.

    Thanks in advance for any help rendered,
  2. Slymenstra macrumors newbie

    Mar 6, 2010
    I'm very new to applescript and I'm working on this same issue at the moment. I've figured out a functional solution to this problem, but I can't find the proper code to express the idea

    My desire is to try to fetch the pixel color in this fashion:
    display dialog Pixel(10, 10)

    In doing further research this is the only solution I've found, which requires an image file and doesn't work on screen colors:

    set imgpath to quoted form of POSIX path of (choose file with prompt "Please choose an image file:" without multiple selections allowed and invisibles)
    set pixelposx to "1.0"
    set pixelposy to "1.0"
    set pyscriptpath to quoted form of POSIX path of (((path to desktop) as Unicode text) & "")
    set command to "/usr/bin/python " & pyscriptpath & " " & imgpath & " " & pixelposx & " " & pixelposy
    set output to do shell script command
    set {red, green, blue} to words 2 through 4 of output
  3. Slymenstra macrumors newbie

    Mar 6, 2010

    OK i think i've got it. This has some extra stuff, but you can trim out whatever you don't need. I've looked and looked for an answer to this, and finally found it by digging through commands.

    The first thing, this solution requires Extra Suites. It's free, but it runs a popup when you run scripts unless you pay 10 USD.

    tell application "Extra Suites" to ES move mouse {10, 120}
    tell application "Extra Suites" to ES click mouse with control
    set myLocation to {10, 120}
    set myColor to {213, 248, 255}
    tell application "Extra Suites"
    	ES move mouse myLocation
    	set coloratpixel to ES pixel color
    	if (coloratpixel is equal to myColor) then
    		ES click mouse with double click
    		set x to item 1 of coloratpixel
    		set y to item 2 of coloratpixel
    		set z to item 3 of coloratpixel
    		display dialog y
    		display dialog z
    	end if
    end tell
    what this does is move your mouse to myLocation, and see if it's equal to myColor. if it is, it double clicks. If it's not, i'm echoing X Y and Z color values into a dialog
  4. jpyc7 macrumors 6502

    Mar 8, 2009
    Denver, CO
    Have you seen Sikuli from MIT Media lab? It's a graphical scripting language where you can automate some GUI interaction. Basically, I think it does some sort of image comparison.

    I'm not sure if it's a tool that can help with what you're doing or not. I've been told by a co-worker that it only works properly if the windows stay in the same location in the screen and that the screen resolution must not change.

    Here's their website:
  5. sammich thread starter macrumors 601


    Sep 26, 2006
    Wow. That's pretty much it. Thanks! :D

    (had a quick look, and it's still pretty rough around the edges, but seriously the best damn thing by a long shot!)

Share This Page