Author Topic: Developer Blog 1 - All about Networking  (Read 43000 times)

0 Members and 1 Guest are viewing this topic.

Offline Vincenzo

  • Flying Squirrel Dev
  • FSE Developer
  • ****
  • Posts: 2352
  • Dutchman living in Allenstein, Ostpreußen.
    • View Profile
    • Flying Squirrel Entertainment
  • Nick: FSE_Vincenzo
  • Side: Union
Developer Blog 1 - All about Networking
« on: November 17, 2012, 06:05:14 pm »
Developer Blog 1 -  All about Networking

Well, that has been one hell of an exciting announcement! The feedback and excitement we have received this week has been overwhelming, and we are proud to say that in just one week we have raised enough to hire our musician to start recording! So be ready for some awesome Drum/Fife/Trumpet and Fiddle tunes coming your way!

Starting today, there will be a weekly dev blog that will keep you up to date with the production of the game.

Next is the Developer's shack. In some developer blogs one of our team members will write a little piece about some of the work they are doing, to give you some more detailed information on Battle Cry of Freedom. This week our main programmer Maxim “Vincenzo” stepped up to explain a little about how we are providing the huge amount of players we have promised!



Networking
I know it, we are insane!, 500 players... It seems impossible right? Well In our first dev blog we will explain some concepts on how we will get this to work.

When designing this network engine we had the following requirements:
  • Normal clans should be able to run servers, on average server hardware and connections.
  • At least double the player count that we could support in M&B: Warband - Napoleonic Wars.
  • It should support Melee/Ranged combat.
  • It must be cheat protected.
  • Environment such as buildings, fences and other scenery must be destructible.
  • Voice chat with other players should be possible.
  • Supporting physics with carts and other moving objects.

With these ideas in mind we went to the drawing board.
The big bottleneck for any multiplayer game with a lot of players, is the server. Figuring the average server would have an Intel Quadcore, and a 100 mbit uplink, we put the requirements for our server to half of these specifications, so a host could run two fully crowded servers on one server box.

First of all we should understand why high player counts are such a challenge, the reason behind it is the exponential amount of data.
In the average FPS game, one player generates about 0.5 Kilobytes of data per second while he is playing the game, this information has to be sent from the server to all other players.

Imagine a 100 player server, that would mean the server receives 0.5 Kilobytes of data from each player, but is sending out 50 Kilobytes per second (0.5 * 100) towards each player..., which would mean the total upstream of the server would be 5000 Kilobytes per second (50 KB/s * 100 players), which is already about 40 megabits total upstream!

Now, imagine a 200 player server, that would come down to (200 * 0.5) = 100 KB/s * 200 players = 20000 Kilobytes per second total upstream which is more than 156 megabit!, which is far more than the average server upstream link... 500 players would even come down to more than 976 megabit upstream which is virtually impossible.

So... how do we do it then?


Only send data when it is actually changed.
A player basically runs a simulation of the world. He gives input such as movement, attacks and other interactions. This input is sent directly towards the server, which processes the input and applies it to the game. These changes are then further sent to the other clients.

Most network engines simply send a constant stream of the latest state-data such as buttons being pressed, directions players are looking at and movement data to all players, for each player, as you might have guessed this creates a lot of redundant data.

In our design we only send changes, for instance if a player turns around, only his turning is sent, and once he stops looking around no further messages are sent for that player, at least until he interacts with the game world again. Same goes for movement, as soon as a player holds forward, only that is sent, as soon as he lets go of the button, that is sent.
This can save enormous amounts of data, because most players are usually not constantly moving/attacking/looking around or interacting with the game world.


Compress data, lossy, to acceptable values.
As you saw before, the amount of data sent is important, saving a single byte per player per network message could come down to 500 bytes already which is half a kilobyte! As such we apply heavy compression on data send from the server towards the clients.
For instance, a player looking around with his mouse moves around 360 degrees. Normally storing the rotation takes a floating point value of 4 bytes...

A single byte can store up to 256 unique values, as such if we map 360 degrees towards 256 values, we might lose precision of the exact rotation, for instance a player might be 1 or two degrees off than what he is looking at in reality, but this is neglectable and virtually impossible to detect for a player looking at another player. Thus we have just saved 75% of the data!

These kinds of compressions are done over the whole engine, voice is also heavily compressed down with the latest voice compression tech.
Most of these kinds of compression techniques have rarely if ever been done to this level of efficiency in other network engines. This is one of the main factors making these insane player counts possible.


Headers are costly, don’t send data too often.
Sending a single network message from the server towards the client takes headers, headers are simply information that routers use to know where a packet comes from and where it should go to.

A average UDP header comes down to about 58 bytes of data, on top of that we add our own 10 bytes header which tells us exactly when a packet was sent and where in the game it should go. As such, sending a message without any data at all already takes 68 bytes of data... thus its very important to send as few messages as possible.
In our design, we basically save up all the data that needs to be sent out to clients, and send it out at a steady 10 times per-second rate, this way we use the headers as efficiently as possible without creating too much delay.


Results.
So, that is well and dandy, but... how does that end up then?
Well, basically a player data packet is between 0 and 7 bytes, depending on the player giving none till all kinds of input. the average during my testing, with an extreme amount of movement changes/strafing and turning around a lot, is around 3 bytes which can be seen as the worst case scenario, as I don't expect all players to move like that all the time..

So in worst case we end up with (500 players * 3 bytes) + 65 bytes header * 10 updates per second = 15680 Bytes per second, or, about 15.3 Kilobytes per second... Times 500 players, it comes down to 7650 Kilobytes per second, or about 59.7 Megabit per second. which is well under 100 megabit upstream of the average server!

We plotted our engine upstream with different player counts in comparison to the average network engine in the following graph:


In Conclusion.
In theory, we could reach much greater player counts, so long as one uses a very good server, with for instance, a 1000 mbit uplink! We announced a player count of 500, as this is what we are confident we can provide, given the current design and previously implemented network engine tests. Progress on the network engine has been great, most of it is working and implemented, We have the confidence to start the live tests soon. For now, using random generated player data yields great results.

If you didn’t realise we were raising money to complete this project head over here to check out our reward tiers here.

We will see you next week!
« Last Edit: December 09, 2012, 11:05:15 pm by Refleax »

Offline Menelaos

  • Brigadier General
  • *
  • Posts: 4000
    • View Profile
  • Side: Union
Re: Developer Blog 1 - All about Networking.
« Reply #1 on: November 17, 2012, 06:10:25 pm »
Great read guys, happy to see some details on how the networking actually plays out. Since you noted that
Quote
At least double the player count that we could support in M&B: Warband - Napoleonic Wars.
And the bug with bonuses lagging servers has gone away. Can we expect to see the player count raised next patch?

Offline Hugh MacKay

  • Donator
  • *
  • Posts: 615
  • Co-founder of the 92nd Gordon Highlanders
    • View Profile
  • Nick: 92nd_Pte_Hugh_MacKay
  • Side: Confederacy
Re: Developer Blog 1 - All about Networking.
« Reply #2 on: November 17, 2012, 06:31:44 pm »
Awesome post! :)
had to read it twice to understand it, but very interesting reading. Looking forward to next blog a lot!
"The most amazing things that can happen to a human being will happen to you if you just lower your expectations."

Offline Gokiller

  • Almost Pride of Ni
  • Major General
  • **
  • Posts: 3913
  • Bydand in the chat lads!
    • View Profile
  • Side: Neutral
Re: Developer Blog 1 - All about Networking.
« Reply #3 on: November 17, 2012, 06:33:07 pm »
Lovely!

Offline Megaberna

  • First Lieutenant
  • *
  • Posts: 754
    • View Profile
  • Nick: Megaberna
  • Side: Neutral
Re: Developer Blog 1 - All about Networking.
« Reply #4 on: November 17, 2012, 06:42:52 pm »
Awesome!

Offline Augy

  • Major General
  • **
  • Posts: 2970
  • Anarchist. Absurdist. Existentialist. Man. Human.
    • View Profile
    • The Royal Recruits
  • Nick: -[TRR]- Cpt. Augy
  • Side: Neutral
Re: Developer Blog 1 - All about Networking.
« Reply #5 on: November 17, 2012, 06:44:05 pm »
You guys got it down!
“Ego is a structure that is erected by a neurotic individual who is a member of a neurotic culture against the facts of the matter. And culture, which we put on like an overcoat, is the collectivized consensus about what sort of neurotic behaviors are acceptable.” -Terence McKenna

Offline Refleax

  • Lieutenant Colonel
  • *
  • Posts: 567
    • View Profile
    • Flying Squirrel Entertainment
  • Nick: Refleax
  • Side: Neutral
Re: Developer Blog 1 - All about Networking.
« Reply #6 on: November 17, 2012, 06:44:43 pm »
Hopefully this should clear some things up for you lot!
May good health be yours.

Offline Harbinger

  • Sergeant
  • *
  • Posts: 89
    • View Profile
Re: Developer Blog 1 - All about Networking.
« Reply #7 on: November 17, 2012, 06:51:53 pm »
Nice read. I really hope player count becomes a major goal of this project. I think I can safely say that the biggest appeal of NW (and now this project) is the large amounts of players in a battle. I'd rather see awesome top-of-the line graphics sacrificed if it meant having more players.

Offline Betaknight

  • Major
  • *
  • Posts: 2614
  • To Protect and To Serve
    • View Profile
  • Nick: Betaknight
  • Side: Union
Re: Developer Blog 1 - All about Networking.
« Reply #8 on: November 17, 2012, 07:12:26 pm »
Next is the Developer's shack.
:O was the name inspired by my shack?

I really liked the idea. It should be interesting Good luck!
« Last Edit: November 17, 2012, 07:18:29 pm by Betaknight »
The first modder for NW.

Offline Hugues

  • Sergeant
  • *
  • Posts: 125
  • A gentleman never tells...
    • View Profile
  • Nick: Hugues
  • Side: Union
Re: Developer Blog 1 - All about Networking.
« Reply #9 on: November 17, 2012, 07:26:53 pm »
Great idea to have developer blogs! Good read.
“I have not failed. I've just found 10,000 ways that won't work.”
― Thomas A. Edison

Paradox Demi-Moderator

Offline Graham

  • Corporal
  • *
  • Posts: 13
    • View Profile
  • Nick: [Nr.24] Graham
  • Side: Neutral
Re: Developer Blog 1 - All about Networking.
« Reply #10 on: November 17, 2012, 08:05:21 pm »
Amazing! Looking forward to next weeks blog.

Graham (is impressed)

Offline Audiate

  • World's Worst
  • Major General
  • **
  • Posts: 9784
  • FREEZE, PUNK!
    • View Profile
  • Nick: the mic rula; the old schoola
  • Side: Union
Re: Developer Blog 1 - All about Networking.
« Reply #11 on: November 17, 2012, 11:45:48 pm »
 It seems there's a couple of geniuses working on this game... ::)
 Great work so far, guys!

Just take it already, take the whole thing
[close]

Offline PsychoPigeon

  • Private
  • *
  • Posts: 49
    • View Profile
  • Side: Neutral
Re: Developer Blog 1 - All about Networking.
« Reply #12 on: November 17, 2012, 11:48:05 pm »
So even if a server has a 200 player cap, it should be ultra lag-free?

Offline Diplex

  • Vital Supporter
  • *
  • Posts: 1430
  • Colonel Edward B. Fowler
    • View Profile
  • Side: Union
Re: Developer Blog 1 - All about Networking.
« Reply #13 on: November 17, 2012, 11:52:23 pm »
Very excited,

Offline DaMonkey

  • King of FSE
  • *
  • Posts: 1677
    • View Profile
  • Nick: King DaMonkey I
  • Side: Confederacy
Re: Developer Blog 1 - All about Networking.
« Reply #14 on: November 17, 2012, 11:53:49 pm »
 Interesting. However I am more interested in the voice communication. I'm concerned as to whether it's server-wide, team-wide, or Direct (like in ArmA, where only those around you hear it).
Did you know that if you use 100% of your brain, you get godlike powers? true story.
Did you know that if you use 10% of received donations, you can release BCoF by now. true story