Writing an Open Source Operating System

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Writing an Open Source Operating System

Postby ex-kgb » Thu Sep 03, 2009 2:00 pm UTC

Hello, I'm writing an open source operating system EDIT: and have already completed certain key components such as multithreading and memory allocation, and I was wondering if anyone was interested in my design ideas. Let me explain.

"Why write another operating system?", you might ask. Linux already runs well and it's free, and there's always BSD. Still, there's one thing missing from all modern operating systems: a more simple design. Code that runs on Linux is ran directly by the CPU: that is, all (well, most) applications are compiled to machine code and executed by your processor. This might seem like a great idea, because of speed benefits, but several things make it bad design. First off, programs ran directly by the CPU pose a security threat. The intel architecture tries to work around this with paging and memory protection, and modern operating systems have to jump through these hoops to make sure everything runs with appropriate permissions. This works, but from a design standpoint, it is far from perfect.

With Valix, the OS I am working on, all code will be ran by an interpreter directly part of the kernel. This allows for use of software protection, not hardware protection, which is much more flexible. Context switching (between ring 0 and ring 3 of the kernel) is not needed, nor is copying data for new processes. This should provide significant performance benefits over traditional interpreters, such as python, that exist outside the kernel.

If you want to learn more, here is the (temporary) website: http://sites.google.com/site/valixsys/

I'm currently looking for developers (C or Asm experience), feedback, criticism, artwork, anything. If you have any questions, please ask.

Cheers,

Xander Vedejas
Last edited by ex-kgb on Fri Sep 04, 2009 4:53 am UTC, edited 1 time in total.

User avatar
lulzfish
Posts: 1214
Joined: Tue Dec 16, 2008 8:17 am UTC

Re: Writing an Open Source Operating System

Postby lulzfish » Thu Sep 03, 2009 9:48 pm UTC

Thoughts:

From Wikipedia:
JavaOS and JX did this with the JVM a while ago and then gave up.
JNODE is currently persuing this with Java, they have a bootable image that sorta works.
Microsoft has done research on it. (Singularity is the research, Midori is the mythical interpreted Windows OS)
Sharp OS and Cosmos are plans to do this in C#, but they are not MS-funded.

If you're going to re-write the kernel, have you thought about what directory hierarchy to use? You'll be re-compiling or re-writing everything anyway, so you can choose whether to follow the FHS or do something more sensible like Windows, Mac, and Gobolinux use.

Are you planning to use the JVM, or a Javascript->machine code engine like V8, or are you going to write your own?

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Thu Sep 03, 2009 10:48 pm UTC

Yes, I've heard of other "Managed Code Kernels", but I am going to write my own interpreter that will be designed specifically for my OS. The interpreter in my OS will also act as the system shell, giving users direct access to exactly the same stuff that programs access.

As for the filesystem heirarchy, I'm not even close to worrying about that yet :) I probably will use a simple solution however, not something cludgy like the windows system or something ancient like the unix system.

User avatar
Earlz
Gets Obvious Implications
Posts: 785
Joined: Sat Jun 09, 2007 8:38 am UTC
Location: USA
Contact:

Re: Writing an Open Source Operating System

Postby Earlz » Fri Sep 04, 2009 2:07 am UTC

ex-kgb wrote:Hello, I'm writing an open source operating system, and I was wondering if anyone was interested in my design ideas. Let me explain.

"Why write another operating system?", you might ask. Linux already runs well and it's free, and there's always BSD. Still, there's one thing missing from all modern operating systems: a more simple design. Code that runs on Linux is ran directly by the CPU: that is, all (well, most) applications are compiled to machine code and executed by your processor. This might seem like a great idea, because of speed benefits, but several things make it bad design. First off, programs ran directly by the CPU pose a security threat. The intel architecture tries to work around this with paging and memory protection, and modern operating systems have to jump through these hoops to make sure everything runs with appropriate permissions. This works, but from a design standpoint, it is far from perfect.

With Valix, the OS I am working on, all code will be ran by an interpreter directly part of the kernel. This allows for use of software protection, not hardware protection, which is much more flexible. Context switching (between ring 0 and ring 3 of the kernel) is not needed, nor is copying data for new processes. This should provide significant performance benefits over traditional interpreters, such as python, that exist outside the kernel.

If you want to learn more, here is the (temporary) website: http://sites.google.com/site/valixsys/

I'm currently looking for developers (C or Asm experience), feedback, criticism, artwork, anything. If you have any questions, please ask.

Cheers,

Xander Vedejas



I actually think the name Valix is taken(I remember seeing it somewhere)

and I would recommend going to http://osdev.org (Troy! where are you!? lol) Read the wikipedia and it's bare bones tutorial. Join the forum when you need help or want to discuss design questions with a forum of nothing but OS devers.(though I suggest you be more specific, and be sure to read the wiki! They don't take kind to noobish questions)

As for the idea.. Sounds very typical. so whats the difference between your project and the hundreds of other people that want an interpreter OS? I can't remember the name, but try googlign for a .Net OS..
My new blag(WIP, so yes it's still ugly..)
DEFIANCE!
Image
This is microtext. Zooming in digitally makes it worse. Get a magnifying glass.. works only on LCD

Goplat
Posts: 490
Joined: Sun Mar 04, 2007 11:41 pm UTC

Re: Writing an Open Source Operating System

Postby Goplat » Fri Sep 04, 2009 3:37 am UTC

ex-kgb wrote:"Why write another operating system?", you might ask. Linux already runs well and it's free, and there's always BSD. Still, there's one thing missing from all modern operating systems: a more simple design. Code that runs on Linux is ran directly by the CPU: that is, all (well, most) applications are compiled to machine code and executed by your processor. This might seem like a great idea,
I'd say that's because it is :)
because of speed benefits, but several things make it bad design. First off, programs ran directly by the CPU pose a security threat. The intel architecture tries to work around this with paging and memory protection, and modern operating systems have to jump through these hoops to make sure everything runs with appropriate permissions. This works, but from a design standpoint, it is far from perfect.
You just give each process a page table containing pointers only to the pages of memory that process is allowed to access; I wouldn't call this "jumping through hoops". Besides, an interpreter must do the same kind of thing - it needs to keep a mapping of variables to addresses. Yours is in fact a more complex design, as an interpreter has to do the same tasks as a virtual-memory OS and more.
Context switching (between ring 0 and ring 3 of the kernel) is not needed, nor is copying data for new processes. This should provide significant performance benefits over traditional interpreters, such as python, that exist outside the kernel.
Benefits that will be far outweighed by the interpretation overhead. You can, of course, get decent runtime speed from an interpreter using techniques like JIT, but the more of that you do, the slower startup becomes. If you want your interpreter to match the speed of compiled programs, you have to do all the same time-consuming and expensive optimizations that compilers do - and how would you like it if firefox took an hour to start? Besides, all this would be very complex, in stark opposition to your stated goal of "a more simple design".

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Fri Sep 04, 2009 4:45 am UTC

I actually think the name Valix is taken(I remember seeing it somewhere)

and I would recommend going to http://osdev.org (Troy! where are you!? ¡This cheese is burning me!) Read the wikipedia and it's bare bones tutorial. Join the forum when you need help or want to discuss design questions with a forum of nothing but OS devers.(though I suggest you be more specific, and be sure to read the wiki! They don't take kind to noobish questions)


The name is taken.. by me. I've posted on osdev much previously (though under the screenname xvedejas) and I've seen you there before too.

Also, I got past the barebones tutorials a LONG time ago. I'm no noob. I already have multithreading and memory management working, and I'm moving on to VESA drivers next. I'm here just to get people interested.

You just give each process a page table containing pointers only to the pages of memory that process is allowed to access


Since the code is managed, I don't need memory protection, the kernel simply needs to knows what memory is for what. I've been able to create a linear, physical-memory allocation scheme that works well. And I don't have separate "processes", everything runs within the kernel, and the illusion of processes is created by having separate threads within the interpreter.

Yours is in fact a more complex design


It is many, many lines of code less (and much more readable IMO!) than the paging and heap code other kernels use. It's not perfect but I do think it is a simpler design.

Benefits that will be far outweighed by the interpretation overhead.


This depends on your point of view. My interpreter will be faster than the average interpreter, I never said faster than machine code. I'm alright with it being a bit slow, if it means a simple managed abstraction for my kernel in the form of an object-oriented interpreter.

User avatar
lulzfish
Posts: 1214
Joined: Tue Dec 16, 2008 8:17 am UTC

Re: Writing an Open Source Operating System

Postby lulzfish » Fri Sep 04, 2009 5:37 am UTC

Can you tell us more about the language and interpreter setup?

User avatar
OOPMan
Posts: 314
Joined: Mon Oct 15, 2007 10:20 am UTC
Location: Cape Town, South Africa

Re: Writing an Open Source Operating System

Postby OOPMan » Fri Sep 04, 2009 11:28 am UTC

Sounds like a cool idea. If only I had some C and Asm skillz :-/
Image

Image

User avatar
zed0
Posts: 179
Joined: Sun Dec 17, 2006 11:00 pm UTC

Re: Writing an Open Source Operating System

Postby zed0 » Fri Sep 04, 2009 12:30 pm UTC

Looking good, if only because you've included colemak by default. :D

Is your shell language going to be the only way of writing programs for Valix, or is there going to be a way to translate a $programming_language program to your shell code?

kmatzen
Posts: 214
Joined: Thu Nov 15, 2007 2:55 pm UTC
Location: Ithaca, NY

Re: Writing an Open Source Operating System

Postby kmatzen » Fri Sep 04, 2009 4:47 pm UTC

ex-kgb wrote:Yes, I've heard of other "Managed Code Kernels", but I am going to write my own interpreter that will be designed specifically for my OS. The interpreter in my OS will also act as the system shell, giving users direct access to exactly the same stuff that programs access.


I'm pretty sure Singularity already does this. The language itself, Sing#, was even designed for this purpose.

User avatar
tetsujin
Posts: 426
Joined: Thu Nov 15, 2007 8:34 pm UTC
Location: Massachusetts
Contact:

Re: Writing an Open Source Operating System

Postby tetsujin » Fri Sep 04, 2009 5:16 pm UTC

ex-kgb wrote:
Benefits that will be far outweighed by the interpretation overhead.


This depends on your point of view. My interpreter will be faster than the average interpreter, I never said faster than machine code. I'm alright with it being a bit slow, if it means a simple managed abstraction for my kernel in the form of an object-oriented interpreter.


I am not alright with my 2.8GHz machine running code at the speed of a 200MHz machine... Just sayin', is all... I had a 200MHz machine and it was just dandy - about ten years ago... People keep suggesting that Moore's Law means that nobody really needs the power their computer has to offer these days. I think the people who hold this opinion simply haven't found enough interesting things to do with their computer.

Here's my thing about memory protection through hardware vs. protection through full virtualization: the only thing full virtualization gets you that hardware memory protection doesn't is the ability to patch bugs in the implementation. For instance if the processor itself had a privilege escalation bug - which has happened in recent times (can't remember the details, but I believe they were able to work around the problem in software).

Every piece of protection you can get by running programs through an interpreter you can also get by running your code on the hardware itself. You can think of this as "jumping through hoops" - but that's just what it takes to deal with real world problems sometimes. Writing a software solution that works reliably and efficiently isn't easy, but it's totally worth it.
---GEC
I want to create a truly new command-line shell for Unix.
Anybody want to place bets on whether I ever get any code written?

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Fri Sep 04, 2009 6:22 pm UTC

I am not alright with my 2.8GHz machine running code at the speed of a 200MHz machine...


Do you run any programs written in python? It will definitely be faster than that. Any in Java? Likely faster than that too. Really, speed won't be as big a deal as you think. And it doesn't even have to be your machine, I'm not making you run it ;)

Looking good, if only because you've included colemak by default. :D


Of course! :)

Is your shell language going to be the only way of writing programs for Valix, or is there going to be a way to translate a $programming_language program to your shell code?


I plan on supporting multiples styles (syntacies) later on, if that is what you mean. I might port the interpreter to another OS like linux, maybe.

Can you tell us more about the language and interpreter setup?


Did you visit the link to my site? It's a work in progress so if you have any specific questions, I'd like to know. All other details will be slowly added to the site.

Sounds like a cool idea. If only I had some C and Asm skillz :-/


Now is a good time to start learning! heh. I actually think C and Asm are horrible languages, but I don't see better alternatives for writing an OS.

User avatar
tetsujin
Posts: 426
Joined: Thu Nov 15, 2007 8:34 pm UTC
Location: Massachusetts
Contact:

Re: Writing an Open Source Operating System

Postby tetsujin » Fri Sep 04, 2009 7:04 pm UTC

ex-kgb wrote:
I am not alright with my 2.8GHz machine running code at the speed of a 200MHz machine...


Do you run any programs written in python? It will definitely be faster than that. Any in Java? Likely faster than that too. Really, speed won't be as big a deal as you think.


I do run programs written in Python. I've even written a couple.

But my computer runs other programs, too - things like Blender, VLC, etc. for which a major speed hit would not be acceptible. Providing a VM and encouraging applications to use it is one thing. Providing a VM and forcing applications to use it is quite another.

As for the speed, if the VM's an interpreter you've got to figure that each instruction is handled by at least a pointer addition and a subroutine call. This alone makes the program execution several times slower. Then you figure all those jumps also blunt the processor's ability to pipeline instructions, and possibly lead to more cache misses - you're talking a serious hit on performance for what I would argue is zero gain.

And it doesn't even have to be your machine, I'm not making you run it ;)


I'm telling you why I think it's a bad idea to build an operating system that specifically does not allow native code execution... My personal machine is just an example.
Last edited by tetsujin on Fri Sep 04, 2009 7:20 pm UTC, edited 1 time in total.
---GEC
I want to create a truly new command-line shell for Unix.
Anybody want to place bets on whether I ever get any code written?

kmatzen
Posts: 214
Joined: Thu Nov 15, 2007 2:55 pm UTC
Location: Ithaca, NY

Re: Writing an Open Source Operating System

Postby kmatzen » Fri Sep 04, 2009 7:18 pm UTC

I'm sort of curious as to the reason for designing a new language for this OS. New languages are cool, but I'm wondering what constructs it actually provides that you can't already get in another interpreted language. You cited speed as a concern. How does your implementation differ from other interpreted languages in order to accomplish this goal? Why not just use an existing language designed for highly concurrent systems such as Erlang?

I'm interested in seeing how this turns out and will differ from Singularity since it sounds like you are creating your own managed language and constructing a kernel with it.

kmatzen
Posts: 214
Joined: Thu Nov 15, 2007 2:55 pm UTC
Location: Ithaca, NY

Re: Writing an Open Source Operating System

Postby kmatzen » Fri Sep 04, 2009 7:48 pm UTC

tetsujin wrote:As for the speed, if the VM's an interpreter you've got to figure that each instruction is handled by at least a pointer addition and a subroutine call. This alone makes the program execution several times slower. Then you figure all those jumps also blunt the processor's ability to pipeline instructions, and possibly lead to more cache misses - you're talking a serious hit on performance for what I would argue is zero gain.


This might be an argument to actually use a trace cache for its original intent. Perhaps the chip makers will adjust to support managed kernels when/if they become mainstream. I could see a managed kernel being useful in the mobile space since processors where many architectures are RISC and do not require the trace cache already to cache decoded micro-ops. Not all mobile procs have MMUs available and most are single core although doing 2-way superscalar is not unheard of such as the new iPhone 3GS (nVidia Tegra is changing things though.) Will be exciting to see what happens!

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Fri Sep 04, 2009 7:51 pm UTC

kmatzen wrote:I'm sort of curious as to the reason for designing a new language for this OS. New languages are cool, but I'm wondering what constructs it actually provides that you can't already get in another interpreted language.


I would say elegance is a big factor. That's why I've based it on smalltalk, which is in my opinion the most elegant current programming language. My designs allow a simple syntax to be used as a procedural, functional, or object-oriented coding style.

You cited speed as a concern. How does your implementation differ from other interpreted languages in order to accomplish this goal? Why not just use an existing language designed for highly concurrent systems such as Erlang?


One thing I want to make clear: this OS really is a hobby. As such, I want to have as much fun as possible, and I want anyone collaborating with me also to have fun. Personally I think trying to implement a language that is already set as some standard would be possibly a larger project than the language itself. I'd like to have freedom in this area, I hope you understand. As for speed, I plan to have certain optimizations (for instance, for tail-recursive functions) and generally a simple design. These things together should give rather decent speeds, in my opinion.

But my computer runs other programs, too - things like Blender, VLC, etc. for which a major speed hit would not be acceptible.


I wouldn't call it "major". Besides, with better design, it's likely people will find a better, possibly faster way of doing things anyways. I don't plan on my OS ever running Blender, to be honest, but I'm confident it could, and well, if things go to plan.

As for the speed, if the VM's an interpreter you've got to figure that each instruction is handled by at least a pointer addition and a subroutine call. This alone makes the program execution several times slower.


Your first sentence is partly true, but I can't say I agree with the second. I plan on having a lot of built-in datatypes and methods for speed reasons. These will be like a C library in the kernel that the interpreter can call. I know of one other person attempting a project like I am; he claims that his OS (which already has a working interpreter, but it hasn't been released yet, called "JmpFarOS") can run interpreted code at 9/10ths the speed as compiled machine-native code. So what I'm proposing all of a sudden doesn't seem so radical.

stephentyrone
Posts: 778
Joined: Mon Aug 11, 2008 10:58 pm UTC
Location: Palo Alto, CA

Re: Writing an Open Source Operating System

Postby stephentyrone » Fri Sep 04, 2009 8:06 pm UTC

tetsujin wrote:As for the speed, if the VM's an interpreter you've got to figure that each instruction is handled by at least a pointer addition and a subroutine call.


I don't see any need for a call. One of the simplest ways to implement an interpreter is basically just a giant switch statement (computed jump).
Last edited by stephentyrone on Fri Sep 04, 2009 8:08 pm UTC, edited 1 time in total.
GENERATION -16 + 31i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.

User avatar
tetsujin
Posts: 426
Joined: Thu Nov 15, 2007 8:34 pm UTC
Location: Massachusetts
Contact:

Re: Writing an Open Source Operating System

Postby tetsujin » Fri Sep 04, 2009 8:08 pm UTC

stephentyrone wrote:
tetsujin wrote:As for the speed, if the VM's an interpreter you've got to figure that each instruction is handled by at least a pointer addition and a subroutine call.


I don't see any need for a call. Even very simple interpreters are often basically just a giant switch statement (computed jump).


True, my mistake. So it's pointer addition (i.e. array subscripting) and jump, followed by another jump, as opposed to a pointer addition and a call followed by a return...
---GEC
I want to create a truly new command-line shell for Unix.
Anybody want to place bets on whether I ever get any code written?

User avatar
Earlz
Gets Obvious Implications
Posts: 785
Joined: Sat Jun 09, 2007 8:38 am UTC
Location: USA
Contact:

Re: Writing an Open Source Operating System

Postby Earlz » Sat Sep 05, 2009 6:07 am UTC

why not use JIT or Static (yet safe) compiling?

I tried doing this with my bytecode project(instead of straight interpreting) and the possible speed benefits can be very great if you rule out a few things possible with machine code. Basically, if you make it so self-modifying code and seperate code and data address spaces, then doing static compile is a trivial thing. It also can give the performance of, or more, but usually not less, than that of say a C compiler. You'll never outsmart the assembly programmer though. lol

It takes a careful architect to craft a language that is known to be safe at the exact time of compiling, and requires careful things in a language if you are to avoid things like bounds-checking and other slow-down factors. It also must have special structures put in to allow for code optimization and the ability to efficiently output efficient machine code.

It's definitely possible, if you can get people use to a Harvard architecture..
My new blag(WIP, so yes it's still ugly..)
DEFIANCE!
Image
This is microtext. Zooming in digitally makes it worse. Get a magnifying glass.. works only on LCD

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Sun Sep 06, 2009 1:02 am UTC

why not use JIT or Static (yet safe) compiling?


Because I want to minimize context switches. I believe if I can eliminate all unneeded context switches I can improve speed greatly.

Rysto
Posts: 1460
Joined: Wed Mar 21, 2007 4:07 am UTC

Re: Writing an Open Source Operating System

Postby Rysto » Sun Sep 06, 2009 2:03 am UTC

ex-kgb wrote:
why not use JIT or Static (yet safe) compiling?


Because I want to minimize context switches. I believe if I can eliminate all unneeded context switches I can improve speed greatly.

That's a wasted effort when you compare that to the overhead of an interpreter.

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Sun Sep 06, 2009 4:01 am UTC

Rysto wrote:
ex-kgb wrote:
why not use JIT or Static (yet safe) compiling?


Because I want to minimize context switches. I believe if I can eliminate all unneeded context switches I can improve speed greatly.

That's a wasted effort when you compare that to the overhead of an interpreter.


I disagree. It's something that happens hundreds of times per second on OSs like linux. The less cpu time spent switching processes, the better; it should be magnified quite a bit. With my current plans, the only sort of context switches are very simple jumps for threading (already implemented) and vm86 mode to do things like change the vesa screen res (work in progress). The former will happen at least a hundred times a second, and the second one is only called when enabling graphics mode or changing the screen res. I want to avoid almost all of the first type.

User avatar
Sc4Freak
Posts: 673
Joined: Thu Jul 12, 2007 4:50 am UTC
Location: Redmond, Washington

Re: Writing an Open Source Operating System

Postby Sc4Freak » Sun Sep 06, 2009 10:43 am UTC

The cost of a hardware context switch on a modern x86 processor is on the order of microseconds. If you perform 1000 context switches a second at a cost of 1µs/switch, you've spent a grand total of 1ms or 0.1% of your time performing context switches. In comparison, a bytecode interpreter will have a far higher overhead. As others have pointed out, you have, at minimum, several instructions of overhead even if you use a big jumptable.

If you have an approximate 1:1 correspondence between bytecode instructions and ASM instructions, you've got an overhead in the range of hundreds of percent. The only way you'll get close to 90% efficiency in your interpreter is if each of your bytecode instructions performs a monumental amount of work. Even JIT'd languages like Java and C# can have trouble reaching 90% performance of an optimised native (eg. C++) program.

fazzone
Posts: 186
Joined: Wed Dec 10, 2008 9:38 pm UTC
Location: A boat

Re: Writing an Open Source Operating System

Postby fazzone » Sun Sep 06, 2009 3:44 pm UTC

Well, even if it doesn't turn out to be as fast as Linux or other kernels, it does sound like an interesting project. I've never done any OS development before, but it is definitely something I'd like to learn about, and I consider myself pretty good if not great at C and ASM, so I'd like to help if I can.
*/

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Sun Sep 06, 2009 4:13 pm UTC

Sc4Freak, I just want to point out that it will be faster than most if not all interpreted languages one might use. You might criticise me for saying this, but I think in a desktop setting that is quite enough. All the heavy lifting will be built-in to the kernel anyways, written in C optimized with the -O3 flag. So I'm confident none of this will be a show-stopper.

Fazzone: take a look at the source code and mess around with it. If you can add some drivers or functionality and make it work, that's great. In any case, you might want to contact me (see 'contact us' on the website).

stephentyrone
Posts: 778
Joined: Mon Aug 11, 2008 10:58 pm UTC
Location: Palo Alto, CA

Re: Writing an Open Source Operating System

Postby stephentyrone » Sun Sep 06, 2009 4:28 pm UTC

ex-kgb wrote:I think in a desktop setting that is quite enough.


It's not, unless you have a very extensive set of system libraries written in native code. Common numerical tasks, like matrix multiplication, FFTs, and convolutions, are absolutely brought to their knees by an interpreted environment (they are often meticulously hand tuned to be able to issue 4 vector instructions per cycle per processor without stalls on modern i386 hardware, so a very high performance interpreter that can handle, say, a byte-code instruction every 5 cycles represents a 80x[1] slowdown on a single core; more on a multicore machine).

To which you say, "who cares about numerical computing in a desktop environment?"

I'll tell you who: everyone. Audio playback depends on the FFT and often convolution as well. Image IO depends on the FFT and convolution and matrix multiplies. Basic image processing algorithms need convolutions and matrix multiplies. Spam filters need matrix multiplies. Etc, etc, etc. All kinds of "basic desktop user tasks" depend on these algorithms.

Which brings us to the catch: these algorithms are quite complex, including quite complex memory access patterns, and impossible to test with any sort of completeness. Do you really want them living in the kernel?

The alternative is to add bytecodes to your interpreted language that provide all of the building blocks for these algorithms at a large enough granularity to maintain some performance, but small enough to test / prove correctness. The downside to this is that it will lead to a blowup in your opcode space, and add a lot of complication to the interpreter.

I'm not saying it's impossible, or even that you shouldn't do it (sounds cool, if nothing else!), just think long and hard about the tradeoffs.

[1] 80x is actually wildly optimistic. I've seen some very simple numerical codes in python run 1300x slower than a naive scalar implementation in C.
GENERATION -16 + 31i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.

ex-kgb
Posts: 47
Joined: Sat Dec 20, 2008 3:55 am UTC

Re: Writing an Open Source Operating System

Postby ex-kgb » Sun Sep 06, 2009 4:50 pm UTC

Which brings us to the catch: these algorithms are quite complex, including quite complex memory access patterns, and impossible to test with any sort of completeness. Do you really want them living in the kernel?


I have no problem with it.

User avatar
spudtheimpaler
Posts: 181
Joined: Thu May 24, 2007 11:57 am UTC
Location: München

Re: Writing an Open Source Operating System

Postby spudtheimpaler » Tue Nov 10, 2009 11:37 am UTC

adriaya wrote:How does open source stay alive? I don't know anyone that has contributed to the open source movement and I am around some pretty technical people. Where do they all hide? In Norway? Toronto? This movement is huge, but who the hell is doing it?

Open source does not necessarily mean hobbyist coders. Often the more famous projects have large teams behind them with some financial backing or sponsorship, whereas many of the hobbyist coders projects will be found mostly incomplete on sourceforge et al.
Fueled by tea and bad maths since 1983...


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 8 guests