Early Direct3D, COM and OpenGL

A friend was asking me recently about John Carmacks 1996 blog post endorsing OpenGL over Direct3D.  Carmack’s post on this subject was a big event in Direct3D and OpenGL’s history because of course John Carmack was a legend in the game industry and DirectX has originally specifically been designed to support DOOM on Windows.  I made the deal with John to have Microsoft port DOOM to Windows on the condition that IF he was satisfied with the quality of the port he would publish it as an Id title.  My first attempt to port DOOM to Windows 3.1 was a failure.  Windows just couldn’t do it, which led to the creation of DirectX in Windows 95 to make porting DOOM possible.  DOOM was the first DirectX game.  Obviously we consulted closely with Carmack on our plans to try to support 3D hardware in Windows 95 which was why it came as a huge shock to the DirectX team in 1996 when Carmack posted the following on his blog.

http://rmitz.org/carmack.on.opengl.html

The effort to create the Direct3D API was initiated when the Windows NT team who had gotten a license from SGI for OpenGL refused to let us ship it with Windows 95.  The OpenGL driver model that the Windows NT was designing for OpenGL at the time was also very primitive, it supported only very low level polygon rasterization hardware.  Most 3D hardware available for the PC at that time actually decelerated 3D rendering performance as a consequence.  Since the OpenGL team refused to work with us on OpenGL support for Windows 95, we acquired Rendermorphics and created our own 3D driver model for Windows 95 called Direct3D.  It was designed at the outset to support the entire range of possible hardware acceleration that we hoped would become prevalent in the market from low-level rasterization all the way up the graphics pipeline to floating point transformation and lighting.  At the time it was a much more advanced driver model for 3D acceleration than anything the Windows NT team was contemplating supporting for Windows NT.  Carmacks principal stated objection to Direct3D in his post was as follows;

“The overriding reason why GL is so much better than D3D has to do with ease of use. GL is easy to use and fun to experiment with. D3D is not (ahem). You can make sample GL programs with a single page of code. I think D3D has managed to make the worst possible interface choice at every oportunity. COM. Expandable structs passed to functions. Execute buffers. Some of these choices were made so that the API would be able to gracefully expand in the future, but who cares about having an API that can grow if you have forced it to be painful to use now and forever after? Many things that are a single line of GL code require half a page of D3D code to allocate a structure, set a size, fill something in, call a COM routine, then extract the result.”

COM… short for Common Object Model was a major strategic push for Microsoft at that time.  We had no official charter to create DirectX at that time, we just did it, but because the DirectX team was a “skunk works project” within the Windows 95 Operating System group where COM support was a major push, we felt strong internal pressure to adopt it in order to have a hope that DirectX might be embraced by Microsoft as a standard API in the future.  If it was NOT COM enabled, the chances of its survival seemed remote.  We weren’t happy about having to COMify DirectX but felt we had little choice.  COM was a convoluted API that was understandably difficult for developers to adopt but it did have a very important property for DirectX… it enabled DirectX API’s to evolve very rapidly without having to worry about backwards compatibility with previous versions of the API.  We knew that whatever we did in 3D at that time was going to be “wrong” and need rapid revision because the consumer market for 3D was so new, nobody had any idea how it would take shape.  We designed Direct3D as a very low level API that OpenGL ( a high level API ) could sit on top of, again with the hope that eventually we would get support for OpenGL internally.  As the first Direct3D documentation posted here shows, OpenGL support was built into the early Direct3D design plan… long before it ever became an issue of public contention.

Linked here is the very first Direct3D API documentation.

http://www.alexstjohn.com/WP/download/D3DOVER.pdf

“The Direct3D HAL is tightly integrated with the DirectDraw HAL and the GDI display driver, giving hardware manufacturers a single, consistent interface to Microsoft graphics APIs, and a long-term, unified driver model for accelerated 3D. Hardware manufacturers need to write only a single driver to accelerate Direct3D, DirectDraw, GDI, and OpenGL. Hardware can accelerate all or part of the 3D graphics rendering pipeline, including geometry transformations, 3D clipping and lighting, and rasterization. The Direct3D HAL has been designed to accommodate future graphics accelerators in addition to those available today.”

Early arguments over Direct3D vs OpenGL were baffling because OpenGL IS NOT a driver model for Windows, it’s a high level 3D API.  OpenGL provides no mechanism for hardware acceleration, a driver model for it ALWAYS has to be a proprietary OS solution.  Direct3D was the Windows driver model for OpenGL hardware acceleration… nothing to fight about right?  Direct3D provided the mechanism for the OpenGL API to interoperate and share graphics processing resources with the rest of the Windows GUI.  So what was Carmacks beef?

1) No OpenGL API for Window 95, Microsoft’s CONSUMER OS version because the Windows NT guys refused to support Windows 95.

2) Absent the high level API support for OpenGL Carmack had to evaluate using the low-level Direct3D driver API directly… which was COM enabled to co-exist with all Windows API’s.

Of course it was a PITA to write a game to a low-level driver API designed to support a wide range of then non-existent 3D hardware, and COM was a PITA to adopt if you were just trying to crank out a game and ship it.  But as troublesome as COM was, it played an important role in the ultimate success of DirectX.  COM provided a way for the API to evolve with rapid releases of new versions of the DirectX API without BREAKING the previous versions of DirectX and the games that depended on it.  If Carmack had adopted Direct3D for Quake, his game would have run without modifications on DirectX 2 through to DirectX 9 all the way to Windows XP because every subsequent generation of DirectX which often included radical API changes from its predecessor would have recognized Quake as a DirectX 2 game and exposed only DirectX 2 API’s for it to use while newer games could use very different DirectX 9 API’s on new Windows OS’s without conflict.  The COM architecture ensured that game developers who targeted DirectX could ship a game and forget about it, reaping profits from its sales years later even as the OS technology evolved rapidly and incompatibly with older games.  This FEATURE of DirectX gave games that used it tremendous longevity without requiring constant maintenance at the expense of more work initially.  That tradeoff wasn’t ideal but also was not without its advantages.  The need for constant backwards compatibility hamstrung the evolution of open API’s like OpenGL that had to try to maintain backwards compatibility over generations at the expense of innovation which is why OpenGL support remains such a scattered and inconsistently supported API across platforms and devices to this day.

What people who like to constantly gripe about Direct3D vs OpenGL today constantly fail to recognize is that Direct3D is the ONLY driver model for 3D hardware acceleration on Windows to this day… even for OpenGL based games!  There are ZERO OpenGL games on Windows that don’t run on Direct3D drivers to get their hardware acceleration.  3D chip vendors like AMD and NVIDIA simply bolt OpenGL API bindings onto their Windows Direct3D drivers to support it.  The consumer problem with applications that do this is that Microsoft does not provide OpenGL API compliance tests for these bindings which means they ship UN-TESTED for compatibility with Windows.  Microsoft doesn’t ship untested drivers for Windows so these bindings are disabled by default unless consumers go to their 3D chip vendors and manually install them resulting in many OpenGL driver related support problems that the Windows WHQL driver certification tests were designed to prevent.

Apple’s recent announcement of a low-level 3D driver API called Metal is their acknowledgment of the need to expose more DIRECT access to 3D hardware acceleration to allow games and high level APIs to evolve faster and take more advantage of recent leaps in GPU performance without being hamstrung by the backwards compatibility constraints of a high level API like OpenGL that DOESN’T have a COM like model for handling API versioning.  In essence Apple is recognizing that they need to embrace a Direct3D like API model for the same reasons DirectX did 15 years ago.  To enable rapid evolution in 3D hardware support.  Even John Carmack recognized this benefit to Direct3D’s design years later.

http://www.tomshardware.com/news/john-Carmack-DirectX-OpenGL-API-Doom,12372.html

“id Software mad scientist and first-person shooter “granddaddy” John Carmack said that DirectX has matured to the point where it’s now a better API than OpenGL. It handles multi-threading better and newer versions manage state better. But he doesn’t have plans in moving to DirectX any time soon, blaming inertia for the studio’s continued use of OpenGL.”

So the understandable conflict over Direct3D vs OpenGL in its early days can be chalked up to a period in history of great uncertainty and rapid change that Direct3D anticipated and was designed to adapt to despite its many early shortcomings.  Carmack, at the time was a Linux fan and held no love for Microsoft (understandably) so his views were certainly shaped by a desire NOT to have his games tied to a specific proprietary platform, nor was he thrilled to have Microsoft assert total control of the PC platform.  Our primary goal in creating DirectX was to ensure that Microsoft’s dominion over the PC desktop didn’t put id Software out of business.  Years later his views about Microsoft’s hegemony over the desktop have proven well founded as Microsoft has often casually made sweeping and destructive changes to the Windows gaming ecosystem in their desire to control more of it for themselves and the XBOX.

http://www.forbes.com/sites/erikkain/2012/07/26/windows-8-is-a-catastrophe-according-to-gabe-newell-valve-hedging-with-steam-on-Linux/

Although I was certainly a “company man” during my career at Microsoft I have suffered many of the same indignities at the hands of capricious technology changes Microsoft has made to Windows gaming over the years in my subsequent ventures.  Here are just a few of my more recent Microsoft stories on the subject for another day.

http://www.extremetech.com/computing/79034-alex-st-john-vista-blows

http://caselaw.findlaw.com/us-4th-circuit/1456200.html

http://web.stanford.edu/class/ee204/Publications/Massive-Microsoft%20EE353%20CasePublisher%202007-registered.pdf

 

 

 

Comments

comments

15 Comments

  1. Yeah. Yeah. Yeah.

    Only proprietary MS solutions could save (IBM PC) world:

    http://www.mesa3d.org/README.WIN32
    (Note: duno if it still work)

    And You where first to provide driver model that could work for variety of hw:

    http://www.mesa3d.org/intro.html
    (First date: 1993)

    And DX games works from DX2.0 to DX9.0c:

    And OpenGL 1.0 do not work on OpenGL 4.4 Compatiblity Profile?
    (Of course it do! And on top of that OpenGL 4.4 is superset of DX 11.2 capabilities …)

    Yes COM did benefit DX to some extend. Yes Microsoft was able to execute DX3D better then OpenGL ARB (in the long run, and only to the DX10/11 after that MS went IE6….)

    And it did pushed things forward. (Also crew behind DX as You say really made best out of not so good situation).

    But You could go the other way. We could have for Windows what we currently have for Linux (Mesa floss drivers!!!), long long long time ago. (By now it would be so much better too)

    • yes actually, Microsoft ensured that ONLY their solutions worked reliably… so that would be unequivocally true. Mesa never stood a chance on Windows which is probably why NOBODY uses Mesa for games on Windows today. …and look what Mesa floss drivers DID for Linux gaming today… it’s HUGE! Oh wait.. it’s not.. what happened there? OpenGL doesn’t WORK consistently across chips in any compatibility profile does it? There must be some reason OpenGL just doesn’t get used for gaming on the PC and there is no gaming market on Linux… what do you think the problem might be if OpenGL works so great? Any ideas? You don’t think that OpenGL’s unstable drivers and inconsistent implementation across chips has any effect on its market viability perhaps? That was the real problem DirectX had to solve… not existence of drivers, stability and consistency of drivers are what made the Windows game market. Developers wanting an easy API to develop for came in a distant second place to CONSUMER demand for reliable games.

      • It, do work consistently.

        Mesa essentially mimic DX by putting common front end before all vendors graphics.

        (And for AMD You will find more and more game devs favouring floss driver… due to consistency)

        I agree on the last part. Tooling, QA for drivers, consistency are above “easy to use”.
        And Mesa cover last 2. First is chicken and the egg of OpenGL in general.

        So Yes alternative was there (Mesa from start was Driver Model too)

        PS Can You also add info why MS never shipped OpenGL wrapper for DX?

        • I don’t know exactly why an OpenGL wrapper for DX was never shipped, I’ll tell you what I recall. There were some differences in the D3D specification that didn’t match OpenGL requirements directly, (that was an artifact of starting with the Rendermorphics code base) particularly around the lighting model as I recall so a correct mapping of the OpenGL API to the D3D API wasn’t possible initially. I also recall that Miscrosoft pissed of John Carmack by demonstrating a QuakeGL wrapper for D3D that ran faster than Quake did directly on hardware that support it, but as any honest 3D developer knows, QuakeGL was never actually an OpenGL based game so it was easier to hack onto D3D. Before I left Microsoft in mid 1997 Gates had declared that D3D was Microsoft’s strategic 3D API and OpenGL support was just for the high end graphics market so that statement by Gates largely ended the OpenGL push within Microsoft… I think that decision had the fortunate consequence of putting the former OpenGL guys to work on Direct3D so that by DirectX 8.0 Direct3D had become robust enough to support OpenGL but Microsoft didn’t care and the OpenGL guys (like yourself) weren’t really about the API, you wanted OpenGL to be the DRIVER MODEL so the OpenGL community (to my knowledge) didn’t bother to produce an OpenGL wrapper for D3D itself, instead the chip makers just baked the OpenGL bindings into their own D3D drivers. Microsoft didn’t want to deal with testing these API’s so the Microsoft WHQL tests that all IHV’s have to conform to have their drivers shipped for Windows requires them to DISABLE these OpenGL bindings requiring the consumer to go direct to the IHV to download them. The debate became irrelevant to Microsoft because nobody but Microsoft can specify a Windows driver model.

          • Well, I know there were some third-party GL-to-D3D wrappers at the time. Such as SciTech GLDirect. Just like their earlier VESA wrapper, the OpenGL wrapper was interesting because not all cards supported VESA/OpenGL at all, or their implementation was flawed.

            As for Mesa… that’s not a driver model. Mesa is a monolithic solution. Yes, there is the Gallium3D project that is more DirectX-like in design (but designed years after DirectX), and *some* drivers in Mesa have been converted to Gallium3D now. Most are still monolithic… and then there are the binary drivers from nVidia and AMD, which are independent of MesaGL altogether (with all due compatibility issues).

  2. “a driver model for it ALWAYS has to be a proprietary OS solution”

    Small note before presenting opposite view:
    I have NO IDEA, if corporate environment from back there could enable floss implementation.
    However if not then, at least today its possible (AMD/Intel, Nvidia for mobile, etc.), its happening, its working.

    Opposite view:
    1) Proprietary solution turn GPU driver into black box. Its impossible to reason about whats inside without help from creators.
    2) Creators usually ( 😉 ) have limited resources devoted to the development. Thus limiting number of users who could get help from 1)
    3) Politics do play role in software development, often forcing suboptimal choices onto devs.
    4) And hardware newcomers need to build their own software solutions. Thus limiting possibility of innovation from outside of established players.

    So FLOSS solution if adopted by MS would help them provide better solution in all aspect, but one:
    Total domination and control of 3D api.

    Arguably keeping front end (shader compiler, DX runtime) proprietary MS could get some benefits of FLOSS solution (e.g. game devs could still spot gpu drivers that misbehave with shaders, direct perf. measures for different technicueq, etc.), without ceding too much control.

    • Hey Microsoft has certainly done enough to screw up gaming on Windows so badly that you can’t blame the community for doing everything in their power to circumvent Microsoft’s control of the platform. That said, I think you’re missing my point. It’s never been about API domination, it’s always about driver architecture domination and Microsoft has total control of that on Windows. NOBODY has the power to circumvent them. You can’t define Floss to integrate well with the rest of the Windows media and graphics system as DirectX does and still be open. ONLY a game wants total fullscreen control of an environment and doesn’t care about integrating with other applications and OS services.. the minute you want the OS to provide other services, adopting a platform isolated graphics API is useless. Does Floss include printer drivers for 3D? Does Floss include streaming video and audio synchronization into 3D scenes? Does Floss enable you to render a browser into a 3D scene without a lot of work? Does Floss provide a mechanism for the background OS to render messages and notifications into a scene? No? Then it’s no use as a driver model for any OS.

      To your specific points;
      1) Turning the GPU into a black box is a GOOD feature of any 3D driver model because the purpose of the driver layer is to expose a consistent interface to INCONSISTENT hardware capabilities. Everybody wants the 3D hardware market to evolve rapidly and for that to happen you need lots of competing architectures. Freezing architecture innovation by EXPOSING the hardware would kill it.
      2) Getting help from lots of monkey’s isn’t necessarily… helpful..
      3) Just as true for open source projects as proprietary ones.
      4) Proprietary driver standards make the market MORE accessible to new hardware because of the black box effect… but both open and proprietary API solutions place constraints on how innovative they can be.

      Just to be clear, I’m not anti-open source, I just think that a lot of folks in the open source community are naïve about how and why proprietary API’s are made and what their benefits really are. I’m willing to take some heat for explaining it to them in the hopes that THEY will make better technology as a consequence. Technologies made by committees end up mediocre and stagnant, successful open source projects generally have visionary leadership that drives them to innovate INSPITE of the communities demands to be egalitarian. Very often it is “evil” corporations like Microsoft and/or Apple that provide the leadership that is necessary for the open source guys to unify around building practical alternatives to proprietary technologies. Unfortunately it’s also pretty rare for open source projects to become commercially viable without an “evil” corporation commercializing them and making them proprietary to some extent in order to profit from them.

      • 1) Its not. You can not debug into driver with “black box”. Also DO note that I’m talking about GPU DRIVER. For Intel and AMD You can have “black box” GPU dirvers and transparent GPU design (both companies release docs, both due to server requirements).

        2) Game devs (largest group of people who will debug into graphic drivers), and OS devs, usually are above avrg. Currently if they work with big corporation, GPU Vendors will be willing to work with them via dev representatives or even dedicated engineers. But there are tons of talented people who work for smaller game studios, floss projects. IRC and/or patch is way more then forum where nobody will respond.

        3) Agree, however as anybody can add what they need downstream… Independent devs can work on whatever they need. Etc. For proprietary code, there is not possibility.

        4) Yes, all current GPU vendors who release FLOSS drivers do design hw specifically for it. On the other hand You have Broadcom example where skilled GPU dirver dev, is making huge strides in writing GPU driver… And he even did not worked on that hw previously. Skilled workforce is more accessible. Good code base is more accessible (at least for bootstrapping development of proper code base). In fact You will find few reverse engineered GPU drivers that work better then their counterparts (in mobile).

        PS OpenGL 4.4 expose more functionality then DX11.2. “Commities” also can have good leadership. If so then its more leadership vs leadership and not floss vs proprietary.
        PSPS Proprietary =/= commercial, FLOSS =/= non-commercial.

        • >OpenGL 4.4 expose more functionality then DX11.2
          who would know? Nobody uses it for consumer applications. No support for it on Apple, no support on mobile, no support for it on consoles, vendor specific support for it on Windows and no games using it. WHY would any game developer target an API… nobody supports?

          • DX11.**2** have same problem.
            And nobody cares that DX11.2 has “No support for it on Apple, no support on mobile, no support for it on consoles, vendor specific support for it on Windows and no games using it. WHY would any game developer target [that] API”

            [end of irony]

            Yes Apple support matter. Its serious roadblocker, and until there is strong case for OpenGL somewhere else (5-10% of relevant markets – gaming), its adoption for newest-shiniest will be hampered. But…

            Its already an option (on Win and Lin) DX12 is not. (And by the end of 2015 even Apple will have OpenGL 4.4 supported…)

            And I’m no way so naive to claim that “cross platform” mean anything to anyone.

            If that conversation go nowhere can I (humbly) ask You if You would be willing to compare DX/Mantle/Metal and OpenGL way of dealing with ‘low overhead’. If I’m not wrong those two “camps” do it differently (and old DX would be included in the first one).

          • http://www.bit-tech.net/news/gaming/2013/06/28/directx-11-2/
            This would be 40% of all new console sales and 100% of all new PC sales WW… THAT is a big market.

            I’d like to do a low-level comparison of the API’s but the exercise appears to be a little premature. The DX 12 approach isn’t out yet, AMD has not been forthcoming with access to Mantel, Metal is just announced so I don’t have much to work with yet.

  3. I wonder how high is Carmack’s IQ?

    • Don’t know the answer to that, I do recall thinking that he and Bill Gates had a lot of quirky personality traits in common and wondering if there was a gene that resulted in people like that with unusual curiosity and perseverance. He and Gates were an interesting combination of fascinated by everything and extremely focused and dedicated to understanding/mastering the things they pursued.

      • I have read your articles on recruiting giants. But I am hungry to hear your stories of yourself, Gates, Carmack, or Gabe, or some ‘giants’ out there, put together. Their obvious intelligence, their curiosity, their perseverance in sth, all that quirks, all that stuff. Would be a great article. I’ll even buy it if you sell it. hahaha

Pingbacks

  1. OpenGL 4.x Initialization in Windows Without a Framework

Leave a Reply

Follow

Get every new post delivered to your Inbox

Join other followers:

%d bloggers like this: