Prototype Discussions

Oct 17, 2007 at 3:35 PM
It's probably a good idea to separate the prototype discussions into a separate thread, so please use this thread to make comments and ask questions on the status of the prototype.

For all developers, please feel free to look over the prototype and give comments/suggestions.

I just uploaded a new version (as of Oct 17, 10:33 AM EST) with a somewhat functional GUI and state management system. At this point, I think the overall component coupling design is a good indicator of what the final version will be like. I may start porting over some the template code, see how it goes.

Coordinator
Oct 17, 2007 at 9:55 PM
I'll try and look over it tonight after work.
Oct 17, 2007 at 10:27 PM
Edited Oct 17, 2007 at 10:28 PM
Sorry about the triple-commit this afternoon... I am really not liking the CodePlex source control clients. I use the command-line utility to add files to source control and commit, then only a handful of the files actually get uploaded.

Any chance of migrating to SourceForge? :)

Anyway, there is a simple scene manager, so the prototype can now demonstrate GUI and camera control using the input event system. The next logical step is to port over some of the other XNATemplate components and see how everything goes. I ported your FreeCamera class. I removed all the Terrain references to it.

As far as the QuadTree component, I think it would be best to implement it as a SceneManager (see ISceneManager.cs in Common and TestSceneManager.cs in the test program). That way, clients can use large terrains with your QuadTree scene manager, or load up interior levels with a BSP scene manager, etc. Let me know what your thoughts are. Since you know much more about your code than I do, I'll let you proceed with the implementation/port of it.

However, I think we should keep the concept of a node-based scene graph. It will make a lot of things easier.

Oh, and let me know if you get a TypeLoadException on the new code. For some reason, after I commited the code, I started to get that exception on the downloaded code. If I use my local source tree, everything is fine, but if I download from CodePlex, it will work on the school computers here but not my home computer. I think it's something with the project files, but I'm not sure. I'm going to take a look into it tonight.
Oct 18, 2007 at 12:32 AM
Nevermind about the TypeLoadException. For some reason, the solution has reverted back to a x64 build.

NOTE: If running the prototype on a x64 machine with a x64 OS (XP 64, Vista 64), make sure the configuration is set to x86 and not Any CPU!
Oct 18, 2007 at 5:49 AM
I have begun writing, what I call a 'design paper', just briefly outlining how the system works, why the decision for using such a system, etc.
I figure, it would be pointless to document QuickStart as it currently is (i.e. Nic's code), if we're definitely moving to this new prototype system, so, I have begun documenting what there is of the prototype (i.e. each component so far, Common, Graphics, GUI and Input)...

I'm fairly new to this, and don't really want to be delegating or anything, but I just want to see if we can get some more design discussion going, really only for the fact it will give me more to write about, and possibly a better understanding of QuickStart.
Coordinator
Oct 18, 2007 at 5:49 AM
I can integrate it, no problem. However I would really like to get the terrain bug fixed first. I'm getting emails from people who would like to use the engine and cannot. Although the recent version should temporarily patch the problem, I got an email today of another terrain problem. They haven't responded to me letting me know if they were using an older version or not (hopefully they are).

I had been contemplating the idea of a scene class as well. Possibly even a seperate scene and sceneManager class. Picture a game with multiple levels, and after a level ends the user wants to pass information (like the player) from that level to a new one. They could simply start up another scene, but not have the manager run it, and then after loading into it they close the first scene, and begin the next scene. Having multiple scenes would be managable through a sceneManager class. I was using this concept on my sceneCreator program I was working on, which got taken over by this project. The problem I had with the scene class is that it needed access to so many things that the game class had. Although with your new framework that may not be the case. We will see.

I really need someone with an older computer working on this terrain bug with me, it is gettin soooo old.
Coordinator
Oct 18, 2007 at 5:55 AM


Charion wrote:
I have begun writing, what I call a 'design paper', just briefly outlining how the system works, why the decision for using such a system, etc.
I figure, it would be pointless to document QuickStart as it currently is (i.e. Nic's code), if we're definitely moving to this new prototype system, so, I have begun documenting what there is of the prototype (i.e. each component so far, Common, Graphics, GUI and Input)...

I'm fairly new to this, and don't really want to be delegating or anything, but I just want to see if we can get some more design discussion going, really only for the fact it will give me more to write about, and possibly a better understanding of QuickStart.


You should look into Doxygen, it is free, and self-documents much of the code. It will setup the UML heirarchy, and links the document automatically. For instance you can click on any class and it will show you what is in it, what each function does, what it derives from, etc...
It would likely save you hours.
Oct 18, 2007 at 6:23 AM
I'd help you with the terrain bug, but as I have an 8800 GTX, the bug doesn't occur on my machine. I have an old 9800 Pro sitting around here, but its AGP so I'd need to put together a new machine, and the card was going bad anyway (hence the reason for the nice upgrade). I vowed to not buy any more ATi cards until they fix their horrible linux drivers. I definitely agree that the known bugs should be fixed before integration.

I would highly recommend Doxygen for the documentation generation. Though, that only handles the actual code documentation and someone will still need to write the high-level documentation. I'll definitely help out with that, once the design is finalized. Charion, I'd like to see that document you're working on. I hadn't expected anyone to actually work with the prototype yet, as it's mostly my personal sandbox and there aren't any other contributors on it yet (except you, apparently). :) Honestly, I wouldn't spend too much time on documenting it. No part of it is actually finalized so it may change significantly.

The way I am picturing the scene managers, I don't believe there will be many dependencies. Each node can implement client-side logic, and the physics simulator will hook directly into it through an interface and pump position/orientation updates into the scene manager. We do need to be careful with message passing versus communication via interface. Interfaces will be more efficient, but they increase dependencies, so I'm not sure yet how much we should rely on message-passing. For something like physics though, I think there is just too much data overhead. Messages would be too much in that case. But some input/audio/etc. I'm not sure. For the production pipeline, we can have a content processor for each scene manager. So, your QuadTree manager could have a custom heightmap importer that builds geometry at compile-time (at least on Xbox), same for a BSP scene manager.

How do you like the GUI model so far? I tried to keep it as object-oriented as possible, and it has window sizers like wxWidgets for layout.

On a side note, yes I realize that the prototype is sorely lacking in comments. I apologize for that, I've just been using it as a sandbox. Once I get something decent implemented in it, I'll definitely take some time to document/comment it. As far as I know (now), Charion's the only one that has looked at it in any depth!
Oct 18, 2007 at 6:49 AM
Wow @ Doxygen! Exactly what I had planned.

I didn't know how it worked at first, so I documented the ENTIRE latest prototype release, which as you know, includes QuickStart aswell.
Not a big deal though.

Judging from the amount of downloads, I'd say there are people trying the prototyped version (Although, you've done well at hiding it, they probably just run the root template :p).

I've been reading as much about geomipmapping as possible, but it is STILL quite above me, I guess I should look into other terrain rendering algorithms/techniques, if you know of anything that might provide a ground for geomipmapping, that would be sweet. (I would really like to contribute to QuickStart as a programmer).
Coordinator
Oct 18, 2007 at 6:53 AM
I think it is great to see a different coding style (mine and yours differ greatly). If I never see something different than my own, I won't know how it stacks up. For instance I've never used Dictionarys in C#. Unfortunately I have been too busy to actually sit down and look over the engine prototype to see if I have any criticisms. As far as the gui goes I definitely like what I see. I'm surprised you're working on the GUI part before XNA 2.0. I guess the part you're working on is the in-game GUI. The editor gui will likely be windows forms, buttons, etc....

When you submit a beta prototype, with some more comments and documentation, hopefully by that point I will have the terrain bug fixed, and will then begin integrating the code. It'll be like a spring cleaning during the integration process, I'll get to make sure that everything I'm integrating is really needed, being used, or if it could be done in a better way.

Because our code styles are so different (which I think is awesome), I think it'd be a good idea for both of us to do code reviews when somebody submits a change. If the code review passes, then we can re-submit as being passed and the feature can go into the next version.

Just my thoughts so far.
Coordinator
Oct 18, 2007 at 6:56 AM


Charion wrote:
I've been reading as much about geomipmapping as possible, but it is STILL quite above me, I guess I should look into other terrain rendering algorithms/techniques, if you know of anything that might provide a ground for geomipmapping, that would be sweet. (I would really like to contribute to QuickStart as a programmer).


I would look at any geomipmapping open-source sample you can find, I'm sure there are a couple out there. If you could find one already written for XNA, that'd be even better, as you could test its performance as well.

Also, are you having the terrain bug? If so, feel free to test and work on that if you feel like it, geomipmapping won't be able to be enabled if we cannot define custom index buffers on old ATi cards, and therefore won't be able to be used at all until this bug is fixed. I can't find any information on why old ATi cards might act differently than all others....
Oct 18, 2007 at 7:06 AM
Yeah, the GUI work is all for in-game GUIs i.e. menus, loading screens, options panels, etc. For any tools, I would highly recommend sticking with WinForms!

Also, if any of you have any suggestions on what features you would like to see in the prototype to get an idea of how things fit together, just let me know! At the moment, I'm proceeding in more or less an ad hoc method just to test things.


Oct 18, 2007 at 7:06 AM
I have an ATi Mobility Radeon x2300 on this laptop.
I don't know if I am having problems rendering, but using the latest (non-prototype) version of QuickStart it runs fine.
Now if you use that same map in all versions of QuickStart, then I had a problem before 0.172.

I have 2 other computers (Both desktop), although have only tested QuickStart on one of them, but it did work. (I'm not sure the computer specifications there, but I can find out later).
Coordinator
Oct 18, 2007 at 7:08 AM
Your ATi is probably new enough that it wouldn't have a problem, but if you'd like to find out, goto Terrain.cs, InitLeafSize(), and change it to 128*128. Run and see if you're missing some terrain, if so, you have the bug....

My guess is you probably won't. I have reports that the Mobility x800 does have the issue.
Coordinator
Oct 18, 2007 at 7:15 AM


shawmishrak wrote:
Also, if any of you have any suggestions on what features you would like to see in the prototype to get an idea of how things fit together, just let me know! At the moment, I'm proceeding in more or less an ad hoc method just to test things.


Most gui I can think of is for editing, anything else I can imagine for "in-game" would simply be cosmetic to whatever demo we put into the engine to show it off. For instance if I had a graphical gui in v0.172 I'd have an arrow possibly to show the wind direction and speed, and possibly another for gravity direction and strength. Those things would likely be a waste of your time, as they'd would do nothing for the framework. Cosmetics should be saved until final versions, unless they're cosmetics to features that will be in the engine itself.

Sometimes I like to think about each component individually, and everything I'd want it to do. For instance, your buttons. What about hovering effects, onHover image that might be different, an onHover sound, an onClick sound (or sound in our prototype period), button special effects like glowing. Different types of buttons too. For instance in the Infection game I had buttons that would change screens as their action, and others would toggle a value between true and false (an on/off button, good for features, like "Sound on/off").

These are just things off the top of my head. I would simply go over all the components and prioritize. Something like buttons likely isn't on the top of our list right now to be made, but the framework should be flexible and clean enough to allow integration for it later.
Oct 18, 2007 at 7:26 AM
It would appear my card suffers the same problem as the x800.
Interesting.
Oct 18, 2007 at 3:30 PM
I don't see a GUI as a cosmetic component. Every game has at least some kind of in-game GUI, whether it be as simple as the main menu screen, or as complex as an MMORPG's character stats windows, inventory windows, etc., or Source's Windows-like GUI in-game. I wouldn't underestimate the importance of a solid GUI to the user experience (both developer and end-user), and it allows developers to concentrate on logic instead of how to draw a GUI.

The interesting thing about what's there so far is that almost everything you mentioned is already possible. If the client wants sound, they can hook into the button's Select event and fire off a sound. They want hovering? Well, text hovering is already there, and they could derive from Button and override the Render method to get the glow (need to make Button mostly virtual). This can also be used to implement different types of buttons. The buttons themselves have no implicit logic behind them. It's completely up to the client to hook in event handlers and do whatever they want when the button is pressed.

Anyway, I agree that the GUI is less important than many other things. It's just a logical testbed for the message/event system. I think I'm going to spend some time on the scene manager. It would be nice to start getting together a working version of it, so then we can get a better idea of how things fit together during game-play, especially how physics talks to the scene manager, and how the audio and input talk to the scene manager, and eventually the network later, etc.

That brings up an interesting point, everything designed should be thread-safe (within reason) and network-aware. Later on, when it comes time to multi-thread the engine and integrate Live networking, the framework should be adaptable to it. We should probably discuss how to go about doing this soon.
Oct 18, 2007 at 5:37 PM
Haven't tested the prototype so far. But shaw wrote (I think it was you) there 's a GameStateManager in development.
That would be something very handy and very badly needed. Maybe one of the next releases of the template could showcase the usage of that manager - just some simple menu structure with the actual game in it's own screen.
And I agree to your last posting, a GUI is something very important to any game.
Coordinator
Oct 18, 2007 at 5:39 PM
I didn't mean GUI in general is cosmetic. When I meant was any specific GUI we put on the screen now, like a wind meter specifically, would be cosmetic. Having the tools in place to put a GUI on the screen is definitely not cosmetic.

As far as threads, I agree completely, in fact, I was thinking the physics component should be threaded. I wish I knew more about how the Networking in XNA 2.0 would be set up so that we could take advantage of that framework in our planning right now.
Oct 18, 2007 at 6:27 PM
Actually, I don't think it matters how Live networking in 2.0 will work. All it really allows you to do at the base level is send messages back and forth. This is the same for any networking system, whether it be raw TCP/UDP or Live. All the networking in XNA will really give you is the ability to send messages on the Live network rather than by themselves on the raw Internet backbone, and not have to worry about creating/destroying sockets. As far an integrating it into the engine, it's essentially equivalent to writing a network component using System.Net and using that.
Coordinator
Oct 19, 2007 at 7:09 AM
Yeah, if we keep the networking design generic it should account for whatever they release.

Also, I created a basic 2D game a few months ago, it has a couple of GUI effects you can use if you'd like. They're difficult to describe, but if you'd like to see them you can get the demo here:
http://www.nfostergames.com/bombdizzle.htm
Coordinator
Oct 22, 2007 at 5:32 AM
Shaw, despite our efforts to let people know your prototypes include older versions of the engine with them, they don't seem to realize it. I think we should remove the original engine from them or possibly comment the included engine version with the build description or something. Reading takes effort, unfortunately for most of the people using our engine, most of them are probably just downloading it and running it as fast as they can just to check it out.
Oct 22, 2007 at 8:20 AM
I was a bit confused at to what to actually use, I got the code from the repository and opened the sln. But since I didn't have C#E I looked around and found a sln under build which loads under VS :) I ran it and was a little puzzled that I didn't see what on the screens on the home page. I then installed C#E and XnaExpress and opened the main sln. I got stumped at there are two versions of the code, so I tried to run this, and now I got the same as the screens (Well almost, got an old gfx x700 so my terrain doesn't look nice at all, but I knew about the terrain bug so no puzzeling there)

Would you consider creating a build structure for your project? Where you seperate everything? Also this will help with the a64 problem mentioned here url=http://www.codeplex.com/XNAEngine/Thread/View.aspx?ThreadId=16569. I can help if you want to, done it a few times for other projects.
Coordinator
Oct 22, 2007 at 2:19 PM
If you want to run the newest version, which should have working terrain for all gfx cards, get it from the 'release' section, it should be version 0.176. If you get anything from the source code section at this point it will be the new engine prototypes, as we are re-structuring the engine framework. If you run the VS8 C# build that is a prototype, the C# express including in the source code is an old version. We are waiting until XNA 2.0 to change our source versioning to something integrated with VS8 (full, not express). Any help is great, we haven't had a lot of integration at this point, so we haven't concentrated a lot on versioning just yet.
Oct 22, 2007 at 3:47 PM
Out of curiosity, LordIkon, why aren't you committing your template changes to the source tree? I think part of the confusion is that I've only been commiting source to the repository, whereas you have you been distributing "releases" that are not part of the source tree.

Also, no offense to the CodePlex site or the Team Suite tools intended, but they can be a real pain to use. I've actually adopted a shared TFS/SVN approach to my local source tree for this project. I use SVN internally to version the source and share code between home/school, then use the Code Plex tools only to commit changes at mile stones to the "real" repository. Honestly, I find SVN to be much more intuitive and useful. I've had the CodePlex client "forget" to upload new source files to the site, and the integrated VS2005 tools are slow and cumbersome. That's why my last commit had three attempts! :)

Sturm, the 64-bit problem was completely unrelated to the source tree structure. I was playing around with the project files, and for some reason Visual Studio decided it would be funny to automatically revert to "Any CPU" instead of "x86". I've since deleted the "Any CPU" build configurations.


Oct 22, 2007 at 4:00 PM
Is the goal to optimize the engine for vast landscapes? Obviously, there will be levels of abstraction here to that the engine can handle different types of scenes, but it's probably a good idea to concentrate on one type, at least initially.
Coordinator
Oct 22, 2007 at 5:06 PM
I forget why I quit committing changes to the repository, I guess I need to get back on that. I will try and do that sometime in the next day or two.

My goal with the engine is to allow the user to choose large 3d model levels OR large terrains if they would like. Basically, if they choose to have vast landscapes, they should be able to do that without hinderance. Of course, HUGE worlds may be a huge task. For now I was thinking large terrains, but maybe not "Elder Scrolls 4" large. As a short term goal, something like 4096x4096 would be great.

I would say concentrating on a terrain type first would be easiest if we're going to start with only one type. As you cannot really deal with worlds that exist on 3d models (other than heightmaps), until you have a physics setup to handle it.
Oct 22, 2007 at 6:38 PM
So you're focusing solely on heightmap based terrains? i.e. no overlapping in the xy plane, including caves, overhanging cliffs, etc.
Coordinator
Oct 22, 2007 at 8:20 PM
Until we have a physics engine I don't see any other way. I've really never done an engine from scratch so I'm feeling my way as I go. Wouldn't overlapping require meshes which would require physics for them? I certainly am not ruling out anything. I would absolutely like to have overlapping, but I figured we'd wait until a physics component was finished.
Oct 23, 2007 at 12:38 AM
Depends on what you mean by physics... if all you need is collision then just do a convex decomposition on the static geometry and use an algorithm like GJK to test for collisions against a player sphere or box. It's pretty straight forward, I even have a GJK implementation in the physics module I'm writing right now, but it still needs a lot of work in general. That way, it doesn't matter what kind of geometry you're using, its all treated the same as long as its convex.
Coordinator
Oct 23, 2007 at 12:54 AM
I'll pretend I understood all of what you just said. Any good links to read up on it?

If you find an easy way for the user to integrate a heightmap with caves, overhangs, etc, or a terrain mesh, and have it integrated with a physics setup for collision, that would be awesome. I mean, that is what an engine is all about right? Having something that is as easy as possible to use, that can do as much as possible and as efficiently as possible. So if you updated the terrain component to allow for this, and gave a small demonstration, that'd be sweet. Let me know what you have in mind in this area.
Oct 23, 2007 at 7:10 AM
I won't pretend I understand most of the dynamics behind huge/unlimited random generating landscapes. So I just have to look at this from an architectural point of view: Does it matter? I mean of cause there should be support for endless and fixed size landscapes, not doubt and also physics. But why does the engine care about which one is used? I think this should be abstracted into a TerrainManager, which is then responsible to load the needed Terrain types, but from an engine perspective this is just an entity which has initialize/update/render. The terrain class is then responsible for the actual hard work of getting the terrain up and showing.

Now physics is something totally different, and can be complex. I think that initially the decision has to be made if the engine should use multi threading or not (It's not something you "just" add later). Going for multi threading seems like the choice of today, thinking multi core PCs and the Xbox. Physics should run in its own thread and just update all entities as often as possible, then once each draw cycle all data from the physics thread is copied into the main engine thread for use in rendering the entities. This will give you the most correct physics behavior possible (of cause depending on the physics implementation) and remove the complexity of physics from the main engine.

Creating caves is more a matter of implementation than actually being part of the terrain. A cave can simply be a static object (let's call it interior) placed on the terrain inside a mountain with the opening facing outward. This way there isn’t actually a cave in the height map, it’s just added later. So there is no difference between a house or a cave, except that the cave is usually behind the terrain and just the opening is visible on the surface. But then again a house could be partially covered by a land/snow slide and then also become a "cave".
Oct 23, 2007 at 2:06 PM
Well the heightmap and physics will need some support for caves, so that they can open up a hole in the heightmap for the cave mouth to come out of.
Coordinator
Oct 23, 2007 at 3:33 PM
Edited Oct 23, 2007 at 3:35 PM
ShawMishrak is going to start on the physics component after he finishes up the prototype framework he is working on, so he can give more details, however if I remember right we will be supporting multi-threading in the physics component.

The terrain engine should support random and pre-made terrains eventually, hopefully endless someday as well.

And if we want caves and overhangs I'm not sure if we'd use a heightmap or an actual mesh for the terrain. I've never setup a terrain that wasn't a heightmap, as I've never created a physics component to support anything special like caves.
Oct 23, 2007 at 4:20 PM
Well, what I tried to say was that there isn't really any difference between a house and a cave. A cave or tunnel are bacially just structures which points into the terrain instead of upward. At least it's one possible implementation.
Oct 23, 2007 at 6:07 PM
Regarding multi-threading, it is my hope to not only keep physics in a separate thread, but also to split the physics across multiple threads. Remember the Xbox can execute 6 hardware threads in parallel on 3 cores.

Sturm/Aphid, in terms of collision detection and response, all that really matters is how the terrain is represented. Either you have static mesh geometry (preferably convex), or a heightmap array that can provide a Z coordinate, given X/Y values (with appropriate interpolation). Placing static geometry on a height map should not be a problem, I agree that placing overhanging cliffs as static geometry on top of a heightmapped terrain would work. I'm not so sure about caves though, unless they always remain above the height of the terrain. Otherwise, you need to do some sort of CSG subtraction to punch a hole in the terrain and insert the cave geometry.

In terms of physics, however, all the static geometry will just be baked into the physics module anyway for efficiency, so it really doesn't matter what the source is. Whether you bake in a heightmap or a set of static geometry, the physics module will be coded such that it won't really matter.
Oct 23, 2007 at 6:13 PM
LordIkon, for technical details on the GJK algorithm, there is a good paper here: http://www.win.tue.nl/~gino/solid/jgt98convex.pdf.

Also, the forums at http://www.continuousphysics.com and http://www.mollyrocket.com are good reading in general.

GJK is a way to detect collisions between convex polyhedra, in such a way that it doesn't matter what the shapes are, as long as you can provide a "support mapping" function, which is just a function that returns the farthest vertex of the shape, given a direction vector. It's really quite an intuitive algorithm, its used in Bullet/BulletX and prevents you from having to implement a separate collision routine for every possible pair of convex primitives (sphere/sphere, sphere/plane, sphere/box, plane/box, box/box, etc.), as well as handling arbitrary convex geometry. Meaning, all you have to do for the game models is create a convex hull for the object, and the collision detector will be able to handle it with no extra code changes.