OK, after spending *many* hours on trying to get my new Samsung U28D590D monitor to work at anything above 256x1440 over just the DisplayPort with my *early* 2013 rMBP (I tried the dual-link DP/HDMI solution, but, though doable, the result is not for daily use), I can now safely say that I have a good understanding of every aspect of the problem and I would like to post a guide here for anyone that might be still struggling with the same problem:
First of all,
forget HDMI, as HDMI 1.4 (the one all MacBooks use) has a limited bandwidth that can only produce up to UHD 3840x2080 @ 30Hz, or FullHD 1920x1080 @ 60Hz.
The DisplayPort 1.1 though (used in Macbook Pro
before late 2013 models, which use DP 1.2), can go a little bit higher than that, but never reach UHD@60Hz (which is a shame, btw, since it was a very expensive peace of hardware). After experimenting a little, the max Pixel clock DP 1.1 can use without issues is around 290 MHz.
After experimenting a little with SwitchResX,
the best combination I could get working is 3360x1890 @ 42Hz. This is 87,5% of UHD (much better than the default 2560x1440 that is applied by OS X when using the DP with an early 2013 rMBP) and 42Hz is marginally acceptable to work at (at least by me), since it is a huge difference from 30Hz, but still below 50Hz (which in my opinion is the minimum *good* refresh rate).
The real trick after that was to make HiDPI work at this refresh rate, and this is when all the deep searching begun. After a lot of fiddling, I came to the conclusion that,
even using SwitchResX, HiDPI modes are only available at the refresh rates OS X considers to be the "native" ones, so even if the 3360x1890@42 Hz combination worked, HiDPI modes where only working for 2560x1440@60 Hz (even the manual ones created with SwitchResX).
The solution was
to manually patch the EDID override file of the monitor (the one SwitchResX patches in order to add resolutions, etc), which in my case specified 2 native resolutions: 3840x2080@60 Hz (which OS X ignored, as the DP cannot handle it) and 2560x1440@60 Hz (which was the one it actually picked, and hence the HiDPI modes where based on this mode).
IMPORTANT: it seems that the U28D590D Samsung monitor has at least 2 variations, since I have model version (DisplayProductID in the override file, which has filename ending with -b80) of 2944 and the RGB patch supplied earlier in this thread has a model version of 2945 (override file ends with -b81). For the patient ones that have read this so far, this means that I cannot just post the magic override file for you to use as-is, since your monitor might be different than mine. What I can do is give you detailed instructions on how to do what I did:
I found
a great EDID editing program called AWEDIDEditor (conveniently for Mac!), you can download it here:
http://www.analogway.com/en/products/software-and-tools/aw-edid-editor/#dl. This was a true life-saver, as I am not very good at hex editing
If you feed this program the
binary EDID of your display, dumped with DarwinDumper (
https://bitbucket.org/blackosx/darwindumper/downloads), you will have a nice GUI with which you can do all the changes you want to your monitors EDID (well, the override of the EDID of course, not the actual EDID in the monitor).
Please note that it is advised to "Restore Factory Defaults" in SwitchResX *and reboot* before dumping the EDID.bin file, so that the file is the vanilla/untampered-with EDID of the monitor.
After you successfully open the EDID binary in AWEDIDEditor, you can find the monitor "native" advertised modes in the "Detailed Data" tab of the application. In this tab, I changed the first (UHD@60 Hz) entry with the 3360x1890@42 Hz one, by using the values from the (tested and working) SwitchResX custom resolution entry:
Pixel Clock: 285.00
H. Active Pixels: 3360
H. Blank: 160
H. Front Porch: 48
H. Sync Width: 32
H. Image Size: 607 (not in SwitchResX, but was the same for both already existing entries, so I kept it the same)
H. Border: 0 (same as above)
V. Active Lines: 1890
V. Blank: 38
V. Front Porch: 3
V. Sync Width: 5
V. Image Size: 345
V. Border: 0
The clock speeds are automatically calculated and are the same with the ones in SwitchResX:
H.Clock: 80.97 kHz, V.Clock: 42.00 Hz
After you make the change, go to the "Hexa Viewer" tab, where you can find the final hex EDID (changed parts will be marked in red, which is good for further fine tuning). Now the only thing that is left to do is convert this hex sequence in a format OS X can understand and apply: copy all of it (I did it by hand, since I couldnt find actual clipboard copy functionality in AWEDIDEditor) in a text editor (Notes/Text Editor, or whatever) and use a space every 8 characters, so that you end up with a sequence of 64 8-character-long strings. Add a < (without the quotes) character at the start (no spaces) and a > character (again, no quotes) at the end and, voila, you have the final string you must copy and paste in Property List Editor (I used the native one supplied with Xcode, I suppose there must also be other alternatives). This must become the value of IODisplayEDID key of the plist monitor override file. I am attaching a template for this, make sure you also change the rest of the properties if they are different for your monitor (especially DisplayProductID). You can also change DisplayProductName, which is the monitor name that is used by OS X system-wide. Also note that, as stated above, the file name might have to be changed, too, my monitor uses
-b80, yours might need
-b81.
Now copy this file to /System/Library/Displays/Overrides/DisplayVendorID-4c2d/ (might need root rights here), cross your fingers and reboot!
When the login screen appears after the reboot, if all went well,
you will notice that you should now be at 3360x1890@42Hz (verify this from the monitors OSD), which is now considered to be the monitors
native resolution.
If something goes horribly wrong and you end up with a blank display or whatever (not happened to me, so I dont know what else could go wrong), you can always use another display (eg. the rMBPs monitor or another external monitor), delete the file you put in the Overrides folder and reboot. This should get you back to a working configuration.
If all works well, all you have to do is add custom
scaled resolutions in SwitchResX (my favorite is 5120x2880, which produces a 2240x1260 HiDPI, the one I am currently working on) and (after a reboot) they should show up as active and should work when selected.
If you want an even higher refresh rate and are willing to sacrifice some more pixels resolution-wise, you can always try your own combinations using SwitchResX to try them first and then following the cumbersome procedure to make the override file.
I hope this will be useful to some people, since it took me around 1 hour to post
