PDA

View Full Version : Program a custom OS?




pchipchip
Apr 27, 2012, 12:37 AM
Is it possible to program my own OS from scratch? Don't tell me that it is a stupid idea or anything. I am just wondering if it is possible and if so, where should I start?



throAU
Apr 27, 2012, 12:41 AM
Yes you can.

You'll need to start off by learning assembly language for your chosen CPU architecture, so that you are able to program low level device drivers to get something to boot.

You'll also need to learn the low level details of PCI, SATA, etc.


It's not a small task - learning assembly language alone will take you quite some time.

I'd certainly recommend doing this using something like VMware (until you know what you're doing) to contain your efforts to a protected, separate environment from your day to day computer.

Low level device programming gives you the power to cause all sorts of data corruption and even hardware problems, if run on physical hardware and you don't know what you're doing.

miles01110
Apr 27, 2012, 01:28 AM
Lots of things are "possible." Whether they're likely is competely different.

MattInOz
Apr 27, 2012, 01:54 AM
Depending on why you want to build your own, you might be better to start like NeXT did by farming Open Source and University Research Projects by build out the bones of the OS.

Apple still does this in order to get Low level parts of the OS, generally employing key project staff to ensure they keep moving forward.

thundersteele
Apr 27, 2012, 02:02 AM
The answer is obviously yes. Someone had to do it in the past, otherwise there would be no operating systems. So it can be done again.

Comeagain?
Apr 27, 2012, 02:07 AM
I wonder if you could use what apple already has for boot code and drivers, and then build from that? I know you couldn't legally, but is it possible?

throAU
Apr 27, 2012, 02:46 AM
Well, given that the core of OS X is darwin, and Darwin is open source, well yes you could, it is even legal. A few people did just that with the PureDarwin and OpenDarwin projects.


I agree it would certainly be a lot easier to start off with something like Darwin or FreeBSD in a virtual machine to play with and modify/break, than trying to start out from scratch without any experience in operating system design.

Back in the 80s when an OS wasn't expected to do much (like DOS), writing one from scratch would be a lot easier.

I think its an example of just how much work and complexity is involved that it took apple several failed attempts (and they ended up buying NEXT back off Jobs to turn it into OS X) to replace the original Mac OS.

IMHO the only reason to write your own quick and dirty "OS" these days is if you want to program the machine to do a single task as quickly as possible. Essentially you're BYPASSING any operating system and doing everything yourself. E.g., many games in the 80s and 90s for stuff like the Amiga, Atari ST, consoles, etc used to do this. Positive: You have no operating system sucking CPU power you could use for your app. Negative: YOU are responsible for all low level device access - no drivers, no graphics libraries, no UI, etc are available. You're on your own.

Otherwise (if you want to run other people's software in a multi-tasking environment) writing an efficient scheduler and memory manager is not a trivial task to undertake. Even Windows, Linux, OS X and FreeBSD are having regular changes and tweaks made to improve their schedulers and memory managers, after several decades of real world use and refinement. And that's before you even get to the user interface stuff.


Not to poo-poo the idea, it's how Linus started out with Linux, and many others created operating systems before him - but to make anything even remotely comparable to OS X (or even Linux) will take many people a long time, even if they're good, experienced programmers.

gnasher729
Apr 27, 2012, 04:44 AM
Is it possible to program my own OS from scratch? Don't tell me that it is a stupid idea or anything. I am just wondering if it is possible and if so, where should I start?

Linus Torvalds did. But then he didn't ask anyone where to start.

adrian.oconnor
Apr 27, 2012, 05:25 AM
Linus Torvalds did. But then he didn't ask anyone where to start.

I think you're wrong about that. Linus was a CS student when he started Linux, and he asked plenty of advice (as any smart person would).

As it happens, you can still read some of his conversations with Andrew Tanenbaum on usenet. Tanenbaum is a professor who specialises in teaching OS design. He even built a teaching OS called Minix. The name Linux is a bastardisation of the Minix name, I guess because Linus was studying Minix at the time.

So here's my advice to OP: If you want to write an operating system, get Andrew Tanenbaum's books on the subject. They're not easy to read, but if you want to write an OS they're going to be a serious help. If you can't make it to the end of those books, you're probably biting off more than you can chew.

mrbash
Apr 27, 2012, 01:07 PM
Yes! You don't have to know Assembly either. You can write an OS in any high level language including Java.

Probably the best place to start would be to take a course on Operating Systems, and if that's not possible, reading a book on Operating Systems. That will give you an idea of what an OS is, and which parts of an OS you want to write.

Then you simply write the different parts of the OS. Compile it for your architecture, and away you go.

It is easier than you think, and it is a fantastic idea. It gives you a great understanding of computers.

Catfish_Man
Apr 27, 2012, 01:34 PM
Yes! You don't have to know Assembly either. You can write an OS in any high level language including Java.

Probably the best place to start would be to take a course on Operating Systems, and if that's not possible, reading a book on Operating Systems. That will give you an idea of what an OS is, and which parts of an OS you want to write.

Then you simply write the different parts of the OS. Compile it for your architecture, and away you go.

It is easier than you think, and it is a fantastic idea. It gives you a great understanding of computers.


I'm curious, how does one write an interrupt handler in Java? Or, for that matter, bootstrap the JVM on their OS if their OS is written in Java?

wlh99
Apr 27, 2012, 01:57 PM
I wonder if you could use what apple already has for boot code and drivers, and then build from that? I know you couldn't legally, but is it possible?

The OS/X kernel (Darwin) is open source and the source code is available from Apple. It would be a good exercise to build it and get it running.

http://en.wikipedia.org/wiki/Darwin_(operating_system)

Here is an Apple support document describing how to obtain and build Darwin.

https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/build/build.html

dmz
Apr 27, 2012, 05:25 PM
You cannot write an OS in a high-level language, sorry mrbash, no-can-do.

You can write a GUI that way, on top of Darwin or any of the free UNIX-like kernels. Is that perhaps what you meant?

Writing an OS from scratch is a gargantuan task - try picking a small target if you wish to pursue this idea seriously - an 8-bit processor with a small assembler and few device drivers - VDT, keyboard, floppy disk and serial port are a good start. I wrote a DOS for the ZX80 way back in the early 80's, and for one person, it was still a huge task, but very, very educational.

Good luck, and keep learning!

:apple:dmz

Anonymous Freak
Apr 27, 2012, 06:49 PM
You cannot write an OS in a high-level language, sorry mrbash, no-can-do.

You can write a GUI that way, on top of Darwin or any of the free UNIX-like kernels. Is that perhaps what you meant?

Writing an OS from scratch is a gargantuan task - try picking a small target if you wish to pursue this idea seriously - an 8-bit processor with a small assembler and few device drivers - VDT, keyboard, floppy disk and serial port are a good start. I wrote a DOS for the ZX80 way back in the early 80's, and for one person, it was still a huge task, but very, very educational.

Good luck, and keep learning!

:apple:dmz

Um, Darwin is written in Objective-C. That is a high-level language. Very little of modern OSes are written in assembly. NONE of the Linux kernel is written in assembly. As long as it is 32-bit, and there is a GNU C compiler for it, Linux will run on it.

truehybridx
Apr 27, 2012, 09:23 PM
Is it possible to program my own OS from scratch? Don't tell me that it is a stupid idea or anything. I am just wondering if it is possible and if so, where should I start?

try taking a look at BareMetalOS, or a small, very small distro of unix... it will require some good knowledge of C, assembly helps too

adildacoolset
Apr 27, 2012, 10:33 PM
okay, you can do it after learning assembly language.A good book is called "The art of assembly language programming" by Randall Hyde. I wouldn't recommend a high-level language as the abstractions would just cause more occupation of size and memory, and less performance.

mrbash
Apr 28, 2012, 10:48 AM
I suspect most of the people claiming that you cannot write an OS in a high level language do not have a formal education in computer Science. And for those who do, shame on you, you should know better.

A first course in OS will shown that the level of the language does not matter, it is only the logic that you implement that matters.

I should also add, that assembly is considered a symbolic language, like object code. The operating system converts Assembly code into machine code, just like the operating system converts object code from a high-level language into machine code. That is to say you can write an OS in assembly, like you can write an OS in object code.

I doubt the OP intends to write a commercial OS. Most likely they want to learn some of the core OS concepts like Process, Scheduling and Memory management.

Also to answer some of the critics of my suggestion that it is possible to write an OS in Java: Java handles signals (the general case of interrupts) quite well.

subsonix
Apr 28, 2012, 10:59 AM
I suspect most of the people claiming that you cannot write an OS in a high level language do not have a formal education in computer Science. And for those who do, shame on you, you should know better.

A first course in OS will shown that the level of the language does not matter, it is only the logic that you implement that matters.


The OS will need to interface directly with the hardware, it must not require a runtime or interpreter as there are none. Certain parts of the OS as the bootloader requires assembly, but you will find it in other parts as well.


The operating system converts Assembly code into machine code, just like the operating system converts object code from a high-level language into machine code.

Wrong. The assembler converts assembly instructions to machine code, the cpu can run the machine code natively. Java byte code requires a Java runtime, which in turn requires an operating system where it can be executed, there is your chicken and egg problem as you are writing the operating system there are no Java runtime.

chown33
Apr 28, 2012, 11:29 AM
http://en.wikipedia.org/wiki/Lisp_machine#Technical_overview
All operating system software was written in Lisp. Xerox used InterLisp. Symbolics, LMI and TI used Lisp Machine Lisp (descendant of MacLisp). With the appearance of Common Lisp, Common Lisp was supported on the Lisp Machines and some system software was ported to Common Lisp or later written in Common Lisp. [underline added]

subsonix
Apr 28, 2012, 11:34 AM
What's your point?


The processor did not run Lisp directly, but was a stack machine with instructions optimized for compiled Lisp.

chown33
Apr 28, 2012, 12:10 PM
What's your point?

It was underlined: All operating system software was written in Lisp. Even if it's compiled to machine code, it's still written in Lisp. Or are you arguing that compilers are forbidden?

Did someone at some point have to write assembler? Maybe, maybe not. It's possible there were existing tools that translated Lisp to other forms, such as assembler, and they used that for bootstrapping. So it's quite possible that no person ever wrote any machine code or even assembler.


Further fanning the flames, see this regarding Java byte code:
http://en.wikipedia.org/wiki/Jazelle

I fully realize this is not a complete JVM in hardware, and still requires software support for some byte-codes.

subsonix
Apr 28, 2012, 12:39 PM
It was underlined: All operating system software was written in Lisp. Even if it's compiled to machine code, it's still written in Lisp. Or are you arguing that compilers are forbidden?


Ok, but so what? C is a high level language explicitly created to write operating systems. I'm not arguing that you have to use assembly to write an operating system, but at least for x86 it's required for the boot loader. But that being said it can often be found in other parts as well, if for no other reason to optimize.

There's quite a lot of resources available on this if you are interested, I feel this is moving on to the absurd.

knightlie
Apr 28, 2012, 02:10 PM
Um, Darwin is written in Objective-C. That is a high-level language. Very little of modern OSes are written in assembly. NONE of the Linux kernel is written in assembly. As long as it is 32-bit, and there is a GNU C compiler for it, Linux will run on it.

Darwin is C & C++.

stackoverflow.com: what-languages-are-windows-mac-os-x-and-linux-written-in (http://stackoverflow.com/questions/580292/what-languages-are-windows-mac-os-x-and-linux-written-in)

Sander
Apr 28, 2012, 02:56 PM
You cannot write an OS in a high-level language, sorry mrbash, no-can-do.

JNode (http://jnode.org/)

subsonix
Apr 28, 2012, 03:33 PM
The kernel is written in assembly though as well as part of the vm, and it uses GRUB as a boot loader.

firewood
Apr 28, 2012, 03:56 PM
A custom OS kernel is easy. It's sometimes done as a project in a graduate or upper division CS course. The drivers are the hard part. Maybe a couple to few dozen man-years if you can find enough documentation on all the hardware and CPU configuration needed. Then the libraries and frameworks on top of that can take hundreds to thousands of man-years. Enjoy.

Usually done in C, with tiny bits of assembly for the stuff that's invisible to the C compiler (process state save/restore, cache and TLB/page table setup, & etc.)

knightlie
Apr 28, 2012, 04:27 PM
JNode (http://jnode.org/)

From the page you linked to:

"JNode is a free, open source Java technology based operating system implemented in the Java language with a very small assembler nano-kernel."

fishcove
Apr 28, 2012, 05:47 PM
...
As it happens, you can still read some of his conversations with Andrew Tanenbaum on usenet. Tanenbaum is a professor who specialises in teaching OS design. He even built a teaching OS called Minix. The name Linux is a bastardisation of the Minix name, I guess because Linus was studying Minix at the time.
...



Wow. That takes me back.

I did a project in University (a loooong time ago :) that involved adding interprocess communications to the Minix kernel

Every change meant recompiling the kernel. On an IBM PC AT with a 286 processor, it took all night! Never did get it finished.

BTW I always assumed both Minix and Linux were variations on Unix.

softwareguy256
Apr 29, 2012, 02:16 AM
as someone who knows operating systems, i can say with a level of confidence that you have no idea what you are talking about.

Yes! You don't have to know Assembly either. You can write an OS in any high level language including Java.

Probably the best place to start would be to take a course on Operating Systems, and if that's not possible, reading a book on Operating Systems. That will give you an idea of what an OS is, and which parts of an OS you want to write.

Then you simply write the different parts of the OS. Compile it for your architecture, and away you go.

It is easier than you think, and it is a fantastic idea. It gives you a great understanding of computers.

ghellquist
Apr 29, 2012, 02:33 AM
Wow. That takes me back..

BTW I always assumed both Minix and Linux were variations on Unix.

Some thoughts. Used to do system software programming professionally, have ported several Os-es to new hardware.

There are a few functions that are difficult to implement in pure c. Traditionally they would require assembly language. But quite often today the compilers support special pragmas allowing you to write interrupt routines directly in c. On small processors ( say 8 bit embedded systems) you can generally write the whole stuff directly in c. Now, if this defines as an operating system is a different thing.

It is also several magnitudes of difference between different os. A small os for an 8bit processor might be a few hundred lines of code. A large system like Mac-Os might be several million lines of code. Both are os-es.

I suggest starting with low level stuff, maybe this might be a solution
http://processors.wiki.ti.com/index.php/MSP430_LaunchPad_%28MSP-EXP430G2%29?DCMP=launchpad&HQS=Other+OT+launchpadwiki

And finally, unix, minix, linux.

No they are not really variations. Definetely minix was inspired by unix and linux was inspired by minix and unix. One goal was to give application programs the same interface as unix. This would allow a programmer to easily port the user programs from one os to another, often by simply recompile the program. This goes omly for the commmand line programs, terminal in osx. Hardware driver programming is very different. The graphical front ends are new additions where the os-es each has taken their own direction. Mac-Os is very different from each and every linux variation in porgramming graphical applications. Even old and new versiona of Mac-OS is quite different in that respect.

Sorry, some ramblings from an old grunt.

Gunnar

throAU
Apr 29, 2012, 03:28 AM
Um, Darwin is written in Objective-C.

No its not. Not even core foundation is written in objective C, and that sits on top of darwin...

Whilst you can get away with writing SOME parts of an operating system in C or other higher level languages, at the ground level, to write drivers for hardware, boot code, etc - you need to know assembler.

Even if you're writing code in C, you often need to know assembler, and/or the low level workings of the device you are dealing with to be able to get things small enough and efficient enough.


If you are trying to write a boot loader, you're never going to manage it in C, let alone Java, lol.... there's simply nowhere near enough space to work with - you wouldn't even be able to fit the JVM in there, let alone your actual code...


And yes, starting out, you'd be better off picking some simple hardware (ideally 8 bit) and starting out with that to get the core concepts nailed down.

If writing an OS was easy, Apple wouldn't have bought NEXT, Linux wouldn't still be a 5+ years behind Windows and OS X in terms of hardware abstraction and user friendliness, and Microsoft would have been toppled many many years ago.


Also: as said, "operating system" is a very generic term.

The commodore C64 had an operating system. DOS was an operating system. Something like Windows NT or OS X is many many times larger, far more complex and provides a lot more services for an application programmer to make use of.

fishcove
Apr 29, 2012, 07:50 AM
...
And finally, unix, minix, linux.

No they are not really variations. Definetely minix was inspired by unix and linux was inspired by minix and unix.
..


I was really just referring to the names, not the OS's themselves 8).

robvas
Apr 29, 2012, 10:25 AM
This page would be a good start:

http://wiki.osdev.org/Introduction

ghellquist
Apr 30, 2012, 08:48 AM
Just found this. Those were the days ...
http://www.youtube.com/watch?v=Q07PhW5sCEk&feature=youtube_gdata_player


I was really just referring to the names, not the OS's themselves 8).
Aah. Agree. Names are definitely inspired by. Unix. There was also a Multics before that which did influence Unix quite a bit.

http://en.wikipedia.org/wiki/Multics

Gunnar

----------

This page would be a good start:

http://wiki.osdev.org/Introduction

Real good starting point. Perhaps a bit too towards "mac osx" or "windows" type os, booting from a hard disc and showing a gui. Not all os-es has hard disc and not all interavt through a gui or even a terminal like interface. Regardless, I found a lot of interesting things at the site. Thank you for the link.

Gunnar

Q-chan
Apr 30, 2012, 07:31 PM
If you are serious to learn about operating system development, I recommend to use the Pintos educational operating system developed by Ben Pfaff at Stanford. It is written in C for x86 architecture and runs in a virtual machine. It comes with great documentation and a set of projects, that guide you from simple tasks, like efficient event (and time) mechanisms to complex things like virtual memory, file system and threading. I used it with great success to teach Operating Systems at a college, and the students loved it.

Look here: http://en.wikipedia.org/wiki/Pintos
and here:
http://www.scs.stanford.edu/10wi-cs140/pintos/pintos.html#SEC_Contents
(the second link goes to the official documentation, which includes the download link)

Have fun!

Manfred

Sydde
Apr 30, 2012, 08:37 PM
Curses, now I will be off spending a week reading up on GNU/HURD, vainly hoping that they might make some progress with it. Modularity sounds like such a wonderful idea.

Catfish_Man
May 1, 2012, 12:19 PM
Curses, now I will be off spending a week reading up on GNU/HURD, vainly hoping that they might make some progress with it. Modularity sounds like such a wonderful idea.

Are they still switching kernels every time they get a userland running?

bearda
May 1, 2012, 02:45 PM
NONE of the Linux kernel is written in assembly.

Really? (http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=arch/x86/lib/atomic64_386_32.S;h=00933d5e992f7a8df394ee347717365925c0f58c;hb=HEAD)

Sydde
May 1, 2012, 04:02 PM
Are they still switching kernels every time they get a userland running?

They have a sort of alpha version running on a Mach. The Viengoos thing looked interesting, but it has not been touched in 3 years. L4 seems to have been too much of a problem, I am not entirely clear why. For a quarter century of work to reach this level, one has to wonder about the real viability of the project, I am thinking some sort of OOP-based global architecture might be more suitable than the daemons scheme.