OpenVMS on IA-32 (32-bit x86)

Quick-Nav Menu

"Windows-NT" is "VMS Reimplemented" (sort of)

Facts:

Confusion and Speculation:

Links:

One "High-Value" Return From "Usenet News"

http://groups.google.com/groups?selm=1997May28.101149.7776%40cmkrnl&output=gplain
From: ########################################
Subject: Re: FX!32, affinity etc.
Date: 1997/05/28
Message-ID: <1997May28.101149.7776@cmkrnl>#1/1
References: <009B44A8.70132C31.4@maxwell.ph.kcl.ac.uk><338A488D.6FC7@videotron.ca>
Organization: Kernel Mode Systems, San Diego, CA
Newsgroups: comp.os.vms

In article <338A488D.6FC7@videotron.ca>,
 jfmezei <"[nospam]jfmezei"@videotron.ca> writes:
> David Cathey (Remove MX to mail) wrote:
>> culminated into OpenVMS.  The sad thing is Cutler had to capitulate
>> to a Windows assimilation of his theoretically sound base O/S.
>> You'd think he would have taken the tenets of procedure calling
>> standards with him as well.
> 
> I think it is quite pretentious of the VMSer to think that Cutler 
> had so much design authority over NT and that he would have taken
> his VMS knowledge with him.

Not pretentious at all.  We're simply aware of facts you apparently 
haven't heard about.  

From the internals point of view there is utterly no question that NT 
is VMS re-implemented.  

> Cutler was told to replace DOS with a real kernel over which Windows
> could run, over which the windows API could run etc etc.

This is off-topic, but you're incorrect here also.  Originally there
wasn't even going to be a Win32 API - 16-bit Windows had not gotten
all that popular when NT was conceived.  NT was originally to be a
follow-on to OS/2, and a cooperative effort with IBM.  But somewhere
along the way Windows got pretty popular, IBM and MS split the beast, 
IBM getting the OS/2 parts and MS keeping the NT parts.

> If you look at the PSION PDA operating system (called EPOC), you'll also
> find many many similarities with VMS. Event Flags, Interprocess
> Mailboxes, shared memory between processes, process priorities, and even
> a utility (SPY) which is the equivalent to SHOW SYS. Its IO system is
> similar to VMS (an equivalent to $ASSIGN with the device name
> determining which driver to use, and $QIO which is more or less
> independent of the device itself.).

That's out at the UI and API level.  We're talking about internal 
similarities.  

> NT is an WINDOWS operating system with modern operating systems
> services. They were implemented with the Windows API mentality.
> Stop thinking that NT is VMS with WINDOWS above it.

No, not "VMS with Windows above it", but a VMS-derived design with 
Windows above it, most certainly. 

> NT differs from DOS in that it has real operating system features, but
> the later are not the exclusivity of VMS.

The "real operating system features" you speak of are at the UI and
API level.  They are not the reasons we consider NT to be a
reimplementation of VMS at the internal level.  

How about:

The scheduler.  (process scheduler in VMS, thread scheduler in NT) 
32 scheduling priorities, divided into the "real-time" (16-31) and 
"variable" (0-15) priority ranges.  identical preemption at ready by 
higher-priority threads; identical quantum and priority boost 
implementations; identical CPU starvation avoidance mechanism to get 
out of priority inversion situations; a null thread for each CPU; 
etc., etc.  

Memory management.  0-7FFFFFFF is per-process, mostly 
user-mode-accessible only; 80000000-FFFFFFFF is systemwide, mostly 
kernel-accessible only.  Functionally identical implementations of 
paging vs. swapping. 

I/O.  I could write a book (in fact, I am), but briefly, IRPs are 
IRPs, UCBs are "device objects", CRBs are "controller objects", ADPs 
are "adapter objects", FDT routines are "dispatch routines", 
EXE$QIODRVPKT is IoStartPacket, StartIO routines are StartIO routines, 
fork routines are DPC routines, ASTs are APCs... etc., etc., etc., 
etc., etc.  

Interrupt handling.  32 levels of interrupts (some simulated but this 
is nevertheless the way the code is written).  IPLs on VMS, IRQLs on 
NT.  In order:  Passive level, APC (AST) Level, Dispatch (fork) level, 
then the IO hardware interrupts, then some "hardware maintenance" 
functions like the hardware timer, IPI, power fail notification, and 
HIGH_LEVEL to block all interrupts. 

Face it, JF, you're wrong.  Worse, you are writing not just in
misunderstanding but in ignorance of the facts.  Please go read
_Showstopper_ and _Inside Windows NT_ (Custer) before opining further
on this subject.
 
########################################

NSR Notes:

  1. I've added some highlighting to the above newsgroup posting but have not changed anything else (including the spelling mistakes)
     
  2. IMHO, when someone develops a useful technology (like VMS) which is not properly marketed due to decisions by people who know nothing amount technology (like upper management at DEC in the late 1980s and all of the 1990s) it is almost a crime against humanity. Unlike some others in the VMS news group, I do not see Bill Gates or Microsoft as evil; likewise I do not see Dave Cutler as a traitor. In the case of DEC management they are definitely guilty of incompetence (read "DEC is Dead, Long Live DEC"). In the case of Cutler I would have made the exact same decision. When a smart company picks up the pieces of a stupid company it is called capitalism.
     
  3. BTW, I never believed VMS was copied into Windows-NT. It was just a starting point for a new code-base at Microsoft. Since then, VMS hasn't changed much (other than being ported to 64-bit Alpha then Itanium) while Windows software has undergone constant development. So I was a little surprised when I received the email described in the next point.
     
  4. In 2010-07-xx I was contacted by the author of this post and asked to remove his name. He states that his view has softened since the post while mine has not. .  over the past 17 years while mine has not. So I replaced his name with 40 octothorpes (#). Here is a copy of his email (name removed).
    I would like to ask you to remove from your site this Usenet article, which I wrote many years ago:

    https://neilrieck.net/docs/Windows-NT_is_VMS_re-implemented.html#USENET

    Not long after that I adopted a more "mature" position. It is certainly true that there are many points of very strong influence of VMS on Windows NT (via, of course, Mica). However it is also true that a lot of things were done differently:  

    - The entire executive object and handle system, for example, unifies a great deal of object management, private and shared objects, security checking, etc. As opposed to having arbitrary small numbers for "event flags", another set of small numbers for "I/O channels", just plain old PIDs for doing nasty things to processes, etc.

    - The wait architecture in NT is far superior to that in VMS. Can wait for many objects of different types at once, wait for 'any' or 'all', timeout argument included in all wait calls so no need to run a separate timer alongside. When you look at just what you can do with events in Windows, for example, the "event flag" stuff in VMS just looks amazingly primitive.

    - The layered driver mechanism is designed in, instead of being something driver writers figured out how to do by stealing pointers to FDT routines (really). Yes, in VMS layered drivers were explicitly provided for in a few areas - storage and terminal class/port drivers come to mind - but that's really it. In Windows it's a much more generalized and formal mechanism.

    - Yes, APCs have some problems compared to ASTs. But OTOH we got real threads in the first release of NT. An AST was always a sort of "one additional thread, not full-featured but better than nothing." NT's threads are better.

    And a few things where they DIDN'T copy, but should have:

    - RMS… Ok, I can live without ISAM files in the OS, but something beyond Unix-style bag-o-bytes files would have been nice.

    - And of course they left "real clustering" behind them too. Btw MS eventually acquired the rights to implement VMS-style clustering in Windows but decided not to go that way.

    - I still miss VMS Backup. NT Backup had this bad habit of throwing up its hands and dying at the first tape read error. (Of course so do the TK50, TK70, etc.; they absolutely will not read past the failure point, so all of VMS Backup's CRCs and redundancy XOR blocks are for naught.)

    - I also miss universal inter-language compatible calling sequences.

    So… yeah, they allowed themselves to use similar designs in a lot of key areas (they didn't even change the name of IRPs) but I really wish I could retract my very general and broadly sweeping statement that "Windows NT is VMS re-implemented." Will you accommodate me? Thanks in advance if yes. :)

            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

An excerpt from Inside Windows NT by Helen Custer

Copyright (c) 1993 by Microsoft Press

Note: In 2003 this out-of-print book was still available from www.bookfinder.com

FOREWORD (By David N. Cutler)

In 1965, I graduated from college with a B.A. in mathematics, a minor in physics, and an overwhelming desire to be an engineer and to build things. So I took a job with DuPont in Wilmington, Delaware, as a materials testing engineer. After about a year of absolute boredom, I was lent to the mathematics and statistics group assigned to construct a computer simulation model for a new foam-making process that the Scott Paper Company was developing. Working with machines that never did what I meant them to was humiliating, but within six months I was hooked, and what I had shunned coming out of school -- computers -- turned into my life's vocation.

Soon after, I transferred to DuPont's engineering department, where I could program full time. DuPont had a small group that built online computer system applications. My real motivation for joining this group was to get closer to computers, and in fact, I wanted to work on implementing an operating system. While in the group, I had the good fortune to work on several stand-alone real-time systems where the project involved writing the central control program that scheduled the various tasks and monitored system activity as well as writing the actual application code.

It soon became apparent that the only way I was going to get the opportunity to work on implementing a real operating system was to join a company that made computers for a business. And so in 1971 I left DuPont for a job in Maynard, Massachusetts, with Digital Equipment Corporation (DEC). As it turned out, this put me in the operating system business for quite some time to come. Little did I know that I would be fortunate enough to develop several operating systems in my lifetime; developing one is a rare opportunity for anyone.

My first operating system project was to build a real-time system called RSX-11M that ran on Digital's PDP-11 16-bit series of minicomputers. At the time, our goals seemed very ambitious. We were asked to build a multitasking operating system that would run in 32 KB of memory with a hierarchical file system, application swapping, real-time scheduling, and a set of development utilities. The operating system and utilities were to run on the entire line of PDP-11 platforms, from the very small systems up through the PDP-11/70 which had memory-mapping hardware and supported up to 4 MB of memory.

I have many fond memories of how RSX-11M took shape, I had a rubber stamp made that proclaimed "Size Is the Goal" and proceeded to stamp every last bit of correspondence to make sure that all the programmers and project managers understood how important it was to achieve our goals. We also learned the power of conditional assembly (high level language use in operating systems was in its infancy at this time), and whenever someone added a feature, we just made it a system-generation option.

While developing RSX-11M, we spent most of our time engineering solutions to memory problems. Because the system had to run in 32 KB, we generated a memory budget that divided available memory equally between the between the operating system and the utility programs. That left a mere 16 KB for utility programs and led to long hours tuning overlay structures to achieve acceptable performance for many of the RSX-11M system programs.

Although RSX-11M had some very stringent size and performance constraints, of the systems I've worked on it was probably the easiest one to develop. It involved re-implementing an existing system but allowed us the freedom to change and subset the programming interfaces as long as programs could be reassembled or recompiled with minimal source-code changes. RSX-11M was introduced in 1973, 18 months after we started building it. It proved to be very successful and helped make the PDP-11 the most popular 16-bit minicomputer of its time.

The PDP-11 provide better price/performance than mainframes, was affordable at the department level, and along with other popular minicomputers of the same era, led to the first wave of "downsizing" in the computer industry. Downsizing was an attempt to "bring down" mainframe applications to the minicomputer systems. Many of the mainframe programs were larger than the PDP-11 could easily accommodate, and almost immediately Digital was up what Gordon Bell has deemed the single most important reason that computer architectures become obsolete: the lack of enough address bits.

Out of this need, the VAX architecture was born, and it became one of the most popular architectures of the late '70s and remained popular throughout the '80s. The VAX architecture provided 32 bits of virtual address space and eliminated the need to wrestle programs into what into what seemed to be an ever-decreasing amount of virtual address space.

My second opportunity to develop an operating system arrived with the VAX. I was very fortunate to be chosen to lead the operating system effort for the VAX-11 architecture, the result of which was the VMS operating system.

VMS was Digital's second general-purpose time-sharing system, developed specifically for the VAX architecture. Because the VAX architecture had grown out of the tremendous success of the PDP-11, however, this time it was mandatory to provide more than source-level compatibility for applications.

Thus the VAX-11 architecture included a PDP-11 compatibility mode which PDP-11 instructions were executed directly by hardware. At that time, it was inconceivable that a single operating system could support more than "compatibility environment". Although it wasn't the best known of the PDP-11 operating systems (amazingly, Digital had no fewer the 10 PDP-11 operating systems at one time or another!), RSX-11M was chosen as the operating system interface that would be emulated in PDP-11 compatibility mode on the VAX. This decision probably didn't make sense to a number of people outside the company, but RSX-11M had the largest number of application development tools, and had the most general-purpose operating system features, supporting multitasking, and had a file system structure that could be compatibly extended. Ultimately, the VAX-11 systems ran the RSX-11M binaries right off the distribution kit; it allowed RSX-11M volumes to be directly mounted and their files to be accessed and shared between RSX-11M compatibility-mode programs and native VMS programs.

From a technical perspective, the biggest mistake we made in VMS was not writing it in a high level language. At that time, we had a group of very accomplished assembly language programmers, some stringent size constraints, and no compiler with the appropriate quality for operating system development. So to ensure that we would ship the system in a marketable time frame, we wrote it in assembly language. Looking back on what happened, it would still be hard to make the decision to write VMS in a high-level language. (Moral: The right thing to do technically isn't always the best thing to do financially.)

Early in the '80s, while minicomputers were busy absorbing mainframe and other new applications, two important technologies were emerging: the personal computer (PC) and workstations. After the VMS project, I spent a few years developing compilers and then led a group that built Digital's first MicroVAX workstation -- the MicroVAX I.

Workstations like the MicroVAX provided individual, high-performance computing for applications such as computer-aided design (CAD), whereas PCs supported business applications aimed at personal productivity, such as spread sheets and word processors -- two very successful early PC products. Although workstations were relatively pricey, personal computers had to be affordable to small businesses.

In order to meet price objectives, the original PCs were built with 8-bit, and later with 16-bit, microprocessors. They were constrained in much the same way RSX-11M had been and required considerable effort on the part of programmers and operating system designers to accommodate their limitations. Hardware resources were so scarce that operating systems existed mainly to handle a few low-level hardware functions and to provide a set of file system libraries. But the personal computer offered something that minicomputers did not -- a market in which independent software developers could sell their programs at a higher volume. As a result, the breadth and variety of applications that run on PCs and exploit their capabilities is truly amazing.

In the mid-'80s, microprocessors gained 32-bit addressing, and workstations were quick to take advantage of this capability. However, because of the very large installed base of personal computers and their applications, it was not easy to simply roll in another computer and then recompile and relink all the application software. End users of PCs simply didn't have the source code for all their programs, and they demanded binary compatibility.

In the summer of 1988, I received and interesting call from Bill Gates at Microsoft. He asked whether I'd like to come over and talk about building a new operating system at Microsoft for personal computers. At the time, I wasn't too interested in working on personal computers, but I thought this would be a good time to meet Bill and discuss what he had in mind. What Bill had to offer was the opportunity to build another operating system, one that was portable and addressed some of the concerns people had about using personal computers to run mission critical applications. Form me, it meant the chance to build another operating system!

Bill finally convinced me that this was an opportunity I couldn't pass up, and in October of 1988, I came to Microsoft and started to build the team that would build the new operating system. I didn't realize it at the time, but this would be the most ambitious operating system project on which I had ever embarked.

Our goals for the system included portability, security, POSIX compliance, compatibility, scalable performance (multi processor support), extensibility, and the ease of internationalization. Of all these goals, by far the one that was hardest to achieve and that had the most profound effect in the structure of the system was compatibility. Hundreds of thousands of PDP-11 systems had been sold, but tens of millions of personal computers were in operation! As if that weren't enough, we needed to compatibly support three separate 16-bit operating environments and add new 32-bit capabilities to free personal computer applications from the same kind of virtual address constraints that had existed for the PDP-11. To top it off, we wanted to support the UNIX standard interface specification called POSIX.

Now almost four years later, we are on the brink of bringing this system, Windows-NT, to market. Helen Custer started work on this book when the operating system design began. As our design has matured, the book has undergone continual change to track the operating system architecture. This has been an arduous task -- keeping up-to-date and writing and rewriting the various chapters of the book as the design evolved. Although it is our design, Helen is the one who has captured the essence of that design and made it understandable to more than just serious operating system implementers. For this, we owe Helen a great debt.

It is impossible to acknowledge all the people who have contributed to the design of Windows NT. I must say that I did not design Windows NT -- I was merely one of the contributors to design the system. As you read this book, you will be introduced to some, but not all, of the other contributors. This has been a team effort and has involved several hundred-person years of effort. Perhaps the most important contribution of all was made by the people who have tested and stressed the system. Without their effort, Windows NT could not have achieved the level of quality that it has achieved.

I hope you enjoy this book about Windows NT as much as we enjoyed designing the system.

Dave Cutler
Director, Windows NT Development

January-2006 Update

This next paragraph came to me last week (Jan-2006) in a personal email from an ex-Intel employee.

"Given your interest in VMS you might find this amusing. In the early 1990's we visited Microsoft to try to ensure that their new OS "Windows NT" would be available on IA32. We met with Dave Cutler, and he was adamant that IA32 was doomed and would we please get lost so he could target Alpha and then whatever 64-bit architecture was certain to replace IA32 by Intel. It was not a polite disagreement; that guy (Cutler) HATED IA32 and wasn't reluctant to transfer his displeasure to IA32's representatives (us). What an ugly business meeting. Smart guy, though."

I asked for a clarification on the phrase "IA32" as it pertains to this quote and was told that it refers to both 486 (which was in production in 1990) and Pentium (586 or P5) which was still in development.

{ note: P6 is the core name for Pentium-Pro which was actually somewhat different from Pentium }

November-2010 Update

This next paragraph came to me last week (Nov-2010) in a personal email from an ex-Microsoft employee.

I worked at Microsoft developing Windows NT from Aug. 1991 to 1998 and know some of the details that are not generally published.  Reading your page, I noticed this text:

"developing Windows-NT (new technology) which will run on multiple target platforms (initially: DEC-Alpha, MIPS, and Intel 386. Later inclusions: Intel 486, Intel Pentium)"

The truth is a bit more convoluted.  The initial project was to develop "NT OS/2" to run on Intel's under-development RISC processor, the i860.  As noted at:

https://en.wikipedia.org/wiki/Intel_i860

with a source of Paul Thurott's book, "NT" actually was derived initially from the codename for the i860, N-Ten.  I heard the same thing in my early years in the NT group.  That was the _only_ target processor in the earliest days, and development was done using cross-compilers on OS/2 PCs and a single shared prototype N-Ten system.

Things turned away from the i860 when it became clear it didn't have some key virtual memory support NT required, and Intel was not willing to add it.  Development was re-targetted to the under-development MIPS R4000 processor, at first using a few R3000-based prototypes with a couple of extra features.  Meanwhile, IBM and Microsoft couldn't come to terms over the future of OS/2, and went their own ways, with IBM pursuing OS/2 2.0 and Microsoft switching NT's focus from OS/2 to Windows.  The lowest levels of the kernel remained unchanged, because NT was designed to have OS "subsystems" implementing specific
semantics, this means a de-emphasis of the OS/2 subsystem and the introduction of a new 32-bit Windows subsystem.  When I joined, all the design docs referred to NT OS/2 but we all thought of it as Windows NT at that point.

The addition of x86 as a NT target happened thanks to a couple of NT engineers doing it on their own time, literally weekends and evenings, because Cutler had less than zero interest.  Once their skunkworks project demonstrated viability, Cutler and others came on board.  The first release of Windows NT (3.1) supported only MIPS and x86, though Alpha was added very shortly thereafter (perhaps a month or two). PowerPC (which you didn't list) was added two years later in 1995.  NT 4.0 supported them all, Windows 2000 ended up supporting only x86, due to market demand and the cost of developing and especially testing on the low-volume processors.

Links

Back to Home
Neil Rieck
Waterloo, Ontario, Canada.