macOS NSBorderlessWindowMask from Scratch

Haven't done that before, but look at the NSWindow documentation for the initWithContentRect methods and the style masks for them.

You can set the Presentation mode in the info.plist, or you can make an NSView (not NSWindow) take over the screen.

I'd suggest starting with Apple's borderless window sample project, RoundTransparentWindow. It's easy to understand if you know the basics, and it has a lot of helpful comments.