Network Design

Nov 26, 2007 at 8:51 PM
Plans for the network sub-engine. Not a lot can be done until the source is updated to support the 2.0 beta, as Live is really the only way forward.
Coordinator
Nov 26, 2007 at 11:06 PM
I agree. Shaw or Sturm should probably update the "current framework" to 2.0. Or if the new framework is going to be much different the current prototype then we need to design that from the ground up.

What you could do is work out some demos which demostrate capability. Such as a program that runs both the server and client. Or a small demo showing communication between client-server, and server authorization of things, like position.

Simple examples work great if you make them someone flexible and independent, it makes them easier to move over to another system.
Nov 27, 2007 at 12:27 AM
We need to finalize the design before we write more code. Chances are the framework is going to be re-written anyway.
Coordinator
Nov 27, 2007 at 12:43 AM
Well then, Mike, I would say you should design a networking system out in UML or at least on paper. Know your requirements from the engine that you'll need from other components, and try and anticipate what other systems will require from the networking and how they'll integrate with it.

Not sure how long the prototype design will take, it's tough as most or all of us have school and/or full-time jobs on the side, and we can only communicate via forums.
Nov 27, 2007 at 7:16 AM
It is not really possible to anticipate what others are going to need. Instead make the system plugable, i.e. if you have a class which transfers data from the client to the server, make sure that it's configurable, either using inheritence (though interface) or templatize the class. There might be some kind of regestering schema you can use in order to define custom messages which can be sent.

If you make it modular/plugable chances are that you make most scenarios possible. But do consider performance at all time, if a perf bug is stored in the networking it will affect every player connected.
Nov 27, 2007 at 5:18 PM
Edited Nov 27, 2007 at 5:19 PM
This is what ive got so far. Ive decided for simplicitys sake the sub system will at this stage only simplify creation and joining of a game, and player + host management. It will also sync entities (have i got the right term?) between players. Any custom things, like scores, messages are not supported directly, but the game can send custom strings and any non engine message is saved to a public buffer to interpreted by the games own implimentation. This may change later on, but I think the syncronisation is a big enough job as-is for the moment.

The design goes something like this:

  • The InitializeCore I dont intend to do anything, except register for messages.
  • A public InitializeNetwork method can be called when the client wants a game. It takes a custom struct containing the search parameters, and other factors such as type (system link, loca, etc). It then searches and decides on the best game. Once joining it throughs a message to that affect. On a side note if no game is found after a length of time (say 3 minutes?) it creates one.
  • Once the join method is complete, SetUpMulitplayerGame is called. This reads information like player names, map etc, and passes them to a special class. This class (not necessarilly written by me, as I dont have a good enough understanding of the other systems yet) initialises the map and graphics systems, along with physics and default entities as a new state, and then serves as an interface between the netwrok sub sytem and the other components. Once this class returns the network state is changed to ready. When all players are ready the game starts and a message is thrown, chanign to the game state
  • Send Data is then called every other frame from the update method (60fps = 30 updates a second, more than enough surely?). This queries the class for any changed entities bound to a local user, and sends out an update message if needed. If the host, once every 30 updates it sends an update message for every enitity, which every client must adopt.
  • Recieve data is then called, and any netwrok messages it interpets and passes on tot eh intermediatry class. If the message is not understood, it is saved to string buffer under the assumption it is client specific.
  • The client can call a leave game method. This removes the player form the session and ends the game state, via the intermediatry class.

This is my curent rough draft. The purpose of the intermediatry class is nthat any framework changes means minimal changes to the network subsytem, as a common interface is applied. There is also the vague possibility the network could be adapted to work with other engines. I hope this is modular enoguh Sturm, but I didnt really understand what you meant by plugable.
Coordinator
Nov 27, 2007 at 5:40 PM
Sounds like a good start. I'd like to see your plans for a server-client relationship and how to run client and server on a single machine.
Nov 27, 2007 at 6:04 PM
Looks good so far. I haven't really thought my way through a lot of it yet, but I have some initial comments:


mikelid109 wrote:

  • A public InitializeNetwork method can be called when the client wants a game. It takes a custom struct containing the search parameters, and other factors such as type (system link, loca, etc). It then searches and decides on the best game. Once joining it throughs a message to that affect. On a side note if no game is found after a length of time (say 3 minutes?) it creates one.


I think create and join should be implemented separately. The client should have the choice of creating a game, or joining a game, not just forcing clients to create games if it can't find a game to join. In addition, the join functionality should allow for clients to either 1) join the "best" game that is found or 2) query for all active games matching the search criteria and allow users to select which game to join.


mikelid109 wrote:

  • Once the join method is complete, SetUpMulitplayerGame is called. This reads information like player names, map etc, and passes them to a special class. This class (not necessarilly written by me, as I dont have a good enough understanding of the other systems yet) initialises the map and graphics systems, along with physics and default entities as a new state, and then serves as an interface between the netwrok sub sytem and the other components. Once this class returns the network state is changed to ready. When all players are ready the game starts and a message is thrown, chanign to the game state


I don't think the network component should be in charge of the game play state. It's up the client to load maps, bring up the graphics/physics/state systems, etc. The network component should only be responsible for communication, let the client handle it how it pleases.


mikelid109 wrote:

  • Send Data is then called every other frame from the update method (60fps = 30 updates a second, more than enough surely?). This queries the class for any changed entities bound to a local user, and sends out an update message if needed. If the host, once every 30 updates it sends an update message for every enitity, which every client must adopt.


Make sure this 30 Hz update clock is configurable. Tie it to an elapsed time, not frame rate.


mikelid109 wrote:

  • Recieve data is then called, and any netwrok messages it interpets and passes on tot eh intermediatry class. If the message is not understood, it is saved to string buffer under the assumption it is client specific.
  • The client can call a leave game method. This removes the player form the session and ends the game state, via the intermediatry class.


I wonder if we can use .NET serialization here. For instance, each custom client message could be marked with the Serializable attribute and conforms to whatever struct format the serializer requires.
Nov 27, 2007 at 8:51 PM
Edited Nov 27, 2007 at 9:09 PM

shawmishrak wrote:

I think create and join should be implemented separately. The client should have the choice of creating a game, or joining a game, not just forcing clients to create games if it can't find a game to join. In addition, the join functionality should allow for clients to either 1) join the "best" game that is found or 2) query for all active games matching the search criteria and allow users to select which game to join.


That shouldnt be hard ot do. Ill add it as a property, and return matched games as a colection. A join game method could be implemeted with the game as a parameter.


shawmishrak wrote:

I don't think the network component should be in charge of the game play state. It's up the client to load maps, bring up the graphics/physics/state systems, etc. The network component should only be responsible for communication, let the client handle it how it pleases.



I disagree. The network needs to be certain of a few things, such as an implemented physics engine. This could be done, if the network hangs untill the client sends a message saying the prep is done. Ill defer to you on this though, as it is possible, and easier for me to code!


shawmishrak wrote:

Make sure this 30 Hz update clock is configurable. Tie it to an elapsed time, not frame rate.



Not a problem. I think update cycles not elapsed time. That way if the physics is runnign slow, the network is not looping wiating for it. This way we only update every few physics computations, which is all that is needed. Only the host should be tied to a time.



shawmishrak wrote:

I wonder if we can use .NET serialization here. For instance, each custom client message could be marked with the Serializable attribute and conforms to whatever struct format the serializer requires.


What would be the benefit you envision with this? Im inclined to keep it as simple as possible, at least for now.

LordIkon, I havent made concrete server client plans yet, but its something im aware of. Im going to mainly use peer-to-peer as that is what Live encourages. Nonetheless, I will be using a server to stop tampering, like someone suggested.

On a side note, ive seen people referring to the framework, and the prototype. As im only just getting involved, can someone explain for me what the current state of the engine is please?
Coordinator
Nov 27, 2007 at 9:34 PM
Edited Nov 27, 2007 at 9:34 PM
Peer-to-peer makes FPS games (with multiple players) very difficult (or am I mistaken?), which is what about 1/2 the games out there are.
Nov 28, 2007 at 5:39 AM

mikelid109 wrote:
On a side note, ive seen people referring to the framework, and the prototype. As im only just getting involved, can someone explain for me what the current state of the engine is please?


As you ca see from this Prototype design, there are still a lot of open issues which LordIkon needs to decide on before we can move forward. I'm still waiting on the issue list being created before I can move forward. This means that the NextG code is on hold at the moment as noone is duing the two base tasks (Creating a Game class and designing/implementing a message system).
Coordinator
Nov 28, 2007 at 5:55 AM
I wasn't aware we were waiting on me. I thought Shaw and yourself had made a small list of requirements for an engine framework to get started.

If you give me a list, I'll make some decisions, I'm getting antsy myself, I figured I was waiting for you guys to have time to flesh something out.
Nov 28, 2007 at 7:41 AM
Nope as Shaw stated you are prob the only one who know who is active and who's not. There's no reason for Shaw to go ahead and create and assign issues to people who aren't going to contribute. Also you are the one who should set the timeframe and adjust the schedule for that.

The list Shaw and I made was just a initial list, you might want to add/remove from it, or move items from one release to another.
Coordinator
Nov 28, 2007 at 8:15 AM
Edited Nov 28, 2007 at 8:16 AM
As far as the main framework, the active people are you (Sturm), myself (LordIkon), Shaw, and Mike. I have some others that are looking for smaller parts once the framework is in place.

Mike will be doing networking.

Shaw will be doing physics (which doesn't have to be ready for prototype, but should be integrated so we don't have to refactor it in). Seems like later on Shaw will want to do optimizations and performance improvements, or will be setting code requirements that will lead to more efficient code, or code that produces less garbage.

You (Sturm) are working on messaging (if I remember correctly), as you've customized the messaging system. I know you wanted the console system (slash command system), and I'm sure you'll have some other stuff later on.

I will likely be working on terrain, quad-tree, A.I., and possibly camera stuff. Of all of those, only the camera system will need to be in first framework prototype.

That is all I know that is "for sure", everything else is up in the air. I can go ahead and assign stuff if that is really what we need. I've been trying to add some simple functionality to template versions to keep people interested, but also because I get people that are wanting to use, or are using the engine but need small things to really use it, like the post we've seen about the character staying above terrain.

The reason I haven't jumped into the new engine design is because it is likely a bit over my head with the little time I have. The engine I designed is the template we've been using, and it is of course....lacking. If I were to jump in and design a new one, it would likely be lacking as well.

The fact that everyone is talking about a prototype signifies to me that the current template isn't worth re-fitting and would be easier to design a new one from scratch. Shaw wrote a decent framework, but it sounds like that one would be too tough to re-fit as well.
Nov 28, 2007 at 9:42 AM
I guess what would make the most value to others out there, isn't new features or bug fixes in the template, but a commitment to moving forward. Of cause there is value in keeping the current users satified, and do bugs and implement new features. But I think that most would be happy to get the new engine into their hands and start on using it. If you feel that you don't have the time for coordinating the project I would suggest that you find another coordinator who can help you here.

Most indie/community projects I've worked on died mostly because there wasn't any good project structure and nobody knew what when what would be done. Also users of the framework stopped using it as it became too much of a hazzle. So consistent planning is just as important as good gfx code (though not as fun to do :))
Coordinator
Nov 28, 2007 at 4:09 PM
As far as other coordinators, I have Shaw, but he is busy for another week or so with school and finals. The only other truly dedicated one (over the last month or so) was yourself. Technically you're a developer, if you feel you would work better as a coordinator the position is yours.
Nov 28, 2007 at 5:20 PM

LordIkon wrote:
Peer-to-peer makes FPS games (with multiple players) very difficult (or am I mistaken?), which is what about 1/2 the games out there are.


I disagree. I thought many were peer-to-peer. This is what live encourages anyway, as there is no easy way to route the traffic to one computer. I will be implementing a simple server, but only to resolve any conflicts. I will have it as the host, and implement it so with send data the game checks to see if it is host. If it is it sends out full updates.
Nov 28, 2007 at 6:09 PM
Edited Nov 28, 2007 at 6:10 PM

mikelid109 wrote:

I disagree. I thought many were peer-to-peer. This is what live encourages anyway, as there is no easy way to route the traffic to one computer. I will be implementing a simple server, but only to resolve any conflicts. I will have it as the host, and implement it so with send data the game checks to see if it is host. If it is it sends out full updates.


The SendData method on LocalNetworkGamer objects (the only way I know of to send packets) has overloads for specifying an individual NetworkGamer as the target. i.e.

    session.LocalGamers[0].SendData(new byte[] { }, SendDataOptions.None, session.Host);

I haven't had time for experimentation, but it seems like a call like this would send a packet (you obviously wouldn't send an empty byte array) from the first local player directly to the game's host.
Nov 28, 2007 at 6:11 PM
LordIkon/Sturm,

Just let me know what you need, and I'll try to get on it next week. I can't guarantee anything this week or weekend, but by the middle of next week I should have plenty of time to get this stuff rolling. I apologize for being more a forum troll than a contributer here lately.

Coordinator
Nov 28, 2007 at 11:07 PM
That is cool, I know how that goes.

As far as what we need, all I know is we all need a base framework to work with. I'm not saying you should create that on your own, I'm just saying that is our requirement. We all need to figure out a starting point I think.
Nov 30, 2007 at 1:13 PM
Edited Nov 30, 2007 at 1:21 PM
Im attempting to follow the current pattern of the engine by placing an Interface in the common section. For the initialize method I need to pass in a struct, but I cant place a struct in an interface. Any suggestions?

Edit: Never mind, Ive broken the struct into a number of public variables. Less elegant but it should work
Nov 30, 2007 at 1:40 PM

shawmishrak wrote:

I don't think the network component should be in charge of the game play state. It's up the client to load maps, bring up the graphics/physics/state systems, etc. The network component should only be responsible for communication, let the client handle it how it pleases.



Mikelid109 wrote:

I disagree. The network needs to be certain of a few things, such as an implemented physics engine. This could be done, if the network hangs untill the client sends a message saying the prep is done. Ill defer to you on this though, as it is possible, and easier for me to code!


I dont know what to do for this section, as I will be going with Shaws suggestion. Does anyone have any ideas on how to implement it?
Nov 30, 2007 at 3:37 PM
Scratch my comment on using the .NET serialization functionality. Apparently the .NET Compact Framework on Xbox does not have this.


Dec 1, 2007 at 3:08 PM
Two things:

  • Ive posted a work-in-porgress interface in the work item. This is a deliberate abstarction as I canhave a non threaded and threaded version hidden behind it.

  • Seriously, does anyone have any ideas on how to pass the initial messages, that tell the client to establish physics, entities, maps etc?
Coordinator
Dec 1, 2007 at 7:33 PM
I believe Shaw started the current message system in the prototype, and Sturm took it over. I could be wrong though.
Dec 1, 2007 at 9:33 PM
Yeah, I wrote the original version that was more or less just a proof of concept then Sturm took it over.

It's hard to say what the exact messages will look like without knowing how the message system operates. Also, keep in mind that these initialization messages will be game-dependent too. Different games will have different requirements here. Try to keep it as general as possible. There will be some information that is general, and other information that is very game-specific.
Dec 4, 2007 at 11:54 AM
One other thing to keep in mind. When designing the single-player network model, make sure there's an option that allows us to run a client/server approach without ever touching Live. As Live isn't available in the XNA Redist, requiring even a local connection (even local gamer profile sign-in) will effectively shoot us in the foot as far as how usable the engine is to the public. If we're going with the "server processes logic/client just handles player input and rendering" approach to single player, this is very important.
Dec 4, 2007 at 5:36 PM
Edited Dec 4, 2007 at 7:39 PM
Can do, will do.

Out of interest can we have a straw poll on network systems please?

Just post and say if your in favour of:

  • Only Live - 0
  • Only Sockets - 0
  • Live, then Sockets at some point - 2
  • Sockets, then Live at some point - 0
  • Something else entirely??? - 0
Dec 4, 2007 at 7:27 PM
Option number 3 :)
Dec 4, 2007 at 8:27 PM
I dont know if this is a fair assessment. Obviously 3 or 4 are better than 1 and 2 in terms of functionality! I would say 3 is fine. I agree that future Sockets support would be good, but in all honesty as long as we have the single-player loop back interface functioning, I'll be happy. I'm not much of a multi-player gamer, lol.
Dec 4, 2007 at 8:50 PM
Each to his own I guess :-)

I only wanted a rough idea as some people dont seem to like Live. It reassuring that people seem to think Live then sockets is fine.
Coordinator
Dec 4, 2007 at 11:03 PM
It may be a good idea to get very basic support for sockets in your initial tests so that you'll understand exactly what type abstraction will be needed if we ever have both, otherwise you may setup a networking system that would have to be largely re-written later to support sockets.

I'm for option 5, support for both at the same time. When I was doing porting, I would work one specific feature until it worked perfectly on both systems, you could certainly do that with networking. If you did both at the same time you could design the networking system with no worry of having to refactor it later.