Skip to main content ↑ ↓
Topic: poor game flow in large maps (Read 39413 times) previous topic - next topic

poor game flow in large maps

I've got a game going on a 1024x1024 map  with several large towns and 8 fast trains, (4 Northbound 4 Southbound) Multiple trams running in all full growth towns (15000 or over). The game has become extremely laggy with no seeming cause. It can happen at the start of a month, two days in or halfway. All not needed processes are shut down and the graphics card is running at 'Turbo'.

The same game played in Experimental suffers no lagginess at all.

Maybe a look should be taken at Knightlys 'New Path Search' to see if it can be incorporated into Standard? Or maybe a new type of search path could be written to suit Standard?
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #1 –
I suspect this is not a pak64-specific issue...

Meanwhile, have you compared the performance of the GDI executable vs. SDL?

Re: poor game flow in large maps

Reply #2 –
Quote from: Isaac.Eiland-Hall – on
I suspect this is not a pak64-specific issue...

Meanwhile, have you compared the performance of the GDI executable vs. SDL?

Yes I have, It doesn't make any noticable difference. I haven't tried it with PAK128 as I only have PAK128 Britain which is a EXP game. I would try PAK128 but it takes weeks to get to the stage where I have PAK64.  But you could be right it may be a Simutrans-Standard problem.
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #3 –
Can you post your savegame?

Re: poor game flow in large maps

Reply #4 –
Quote from: gerw – on
Can you post your savegame?
Ok posted to http://simutrans-germany.com/files As Colin's Standard 2611.sve
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #6 –
I have just tried Colin's save game.

There was a brief period of reduced responsiveness immediately after loading, but soon afterwards the game ran smoothly. When I tried to add a halt or adjust certain convoy's schedule, responsiveness was reduced again shortly afterwards for quite some time.

So, probably the reduced responsiveness is caused by the re-routing of all existing ware packets in one single world step after set_schedule_counter() is triggered by actions like halt construction/deletion or schedule changes.

Re: poor game flow in large maps

Reply #7 –
Quote from: Knightly – on
There was a brief period of reduced responsiveness immediately after loading, but soon afterwards the game ran smoothly. When I tried to add a halt or adjust certain convoy's schedule, responsiveness was reduced again shortly afterwards for quite some time.

So, probably the reduced responsiveness is caused by the re-routing of all existing ware packets in one single world step after set_schedule_counter() is triggered by actions like halt construction/deletion or schedule changes.
I've noticed the same, while trying Colin's game. So it's the rerouting of goods, which makes the game laggy...

Re: poor game flow in large maps

Reply #8 –
Quote from: Knightly – on
I have just tried Colin's save game.

There was a brief period of reduced responsiveness immediately after loading, but soon afterwards the game ran smoothly. When I tried to add a halt or adjust certain convoy's schedule, responsiveness was reduced again shortly afterwards for quite some time.


Reduced by quite a lot and when building a game, because one is continually adding halts or new schedules, it happens with repeated monotony. Every few game days actually, and not counting Auto Save which I've extended to every 3 months. Brave little Aussie fellow that I am. ;D ;D ;D
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #9 –
Quote from: gerw – on
I've noticed the same, while trying Colin's game. So it's the rerouting of goods, which makes the game laggy...

Gerw, is it possible to spread the load of re-routing ware packets across a few steps?

Quote from: Colin – on
Reduced by quite a lot and when building a game, because one is continually adding halts or new schedules, it happens with repeated monotony. Every few game days actually, and not counting Auto Save which I've extended to every 3 months. Brave little Aussie fellow that I am. ;D ;D ;D

Colin, I understand how poor the responsiveness would be when you continue to add/remove halts or adjust schedules, because I have a similar PC specifications as yours ;) The game almost freezes during such global re-routing.

Re: poor game flow in large maps

Reply #10 –
Quote from: Knightly – on
Gerw, is it possible to spread the load of re-routing ware packets across a few steps?
It's currently spread across 32 steps, see spieler_t::step. But maybe this number should depend on the time, the rerouting needs.

Re: poor game flow in large maps

Reply #11 –
Quote from: gerw – on
It's currently spread across 32 steps, see spieler_t::step. But maybe this number should depend on the time, the rerouting needs.

Thanks a lot for pointing out. :) It is quite incredible that after spreading re-routing across 32 steps, the game is still so unresponsive.

Re: poor game flow in large maps

Reply #12 –
Quote from: Knightly – on
Thanks a lot for pointing out. :) It is quite incredible that after spreading re-routing across 32 steps, the game is still so unresponsive.
But one can think of a worst case, where all the 'expensive' stations (with many waiting goods) are rerouted in one step...

Re: poor game flow in large maps

Reply #13 –
At least, there wasn't this problem in 99.17.1.
But 100.0 and after versions have this problem.

I heard many people saying 100.0 and after versions are too heavy.

Re: poor game flow in large maps

Reply #14 –
My observations with Colin's savegame:
1) After loading of the savegame, and also after deployment of a new schedule or a new goods category for an existing schedule, ST becomes unresponsive for some seconds, with down to 2(!) fps and 100% cpu/core utilization. => ST seems to recalculate routing for all goods packets, not just for the goods categories served by the new schedule/convoy. This should be fixable by filtering before re-routing.
2) The previous observation especially applies to the change of routing in an isolated network, which seems to still lead to re-calculation of routing on the whole map. For isolated freight lines/networks, it may be useful to limit the refresh, e.g. by walking through the graph of stations *and* connected convoys, also filtering(!) by goods category (see above), marking the stations (also convoys due to their contents?) for refresh, and refreshing only the marked ones in the next loop (removing the marker here again).
1)+2) could be changed as described (or by other means), but a large, fully interconnected P+M network will still not benefit directly from such changes, only indirectly by optimized routing for certain other cases. Further spreading of re-routing calculation may help.
3) side-note: station coverage setting in the savegame is 3 (=> 49 tiles to check at P+M+G generation), which doesn't matter for re-routing, but the additional calculation effort for newly generated p****engers+mail increases overall load.

Re: poor game flow in large maps

Reply #15 –
I have just read whoami's post and it seems to me that he has hit the nail on the head, as they say. If his suggestions can be implemented we should see a, not vast, but quite substantial in crease in game flow. (Incidentally my FPS went down to 1)

I could set the station coverage back to 2 but surely that would lead to an increase in Trams/Tram lines/Stations, creating even more route finding?

I have question re this particular game.

modnote by whoami: this part is now discussed in http://forum.simutrans.com/index.php?topic=3203.msg31352#msg31352:
 I have a p****enger train (Not shown in the uploaded game)  Which travels between two stations only. It leaves one station with over 3000 pasengers and arrives at it's destination where they all disembark. It makes a loss of over 41 thousand (not in travel costs) when the p****engers leave. It returns home with over 300 p****engers and makes a profit of over 22k when the p****engers leave. Have I got a setting in wrong in the config.tab or is this another bug?
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #16 –
As I wrote, my suggestions (if they matter at all) will not help (much) in the case of the p****enger/mail network.

Quote from: Colin – on
I could set the station coverage back to 2 but surely that would lead to an increase in Trams/Tram lines/Stations, creating even more route finding?
Directly, it will only increase routing calculation effort to a smaller extent, because of the larger number of stations ("hops"). But usually, you will have more different lines, more interconnection points and more transfers, and that makes a big difference.

Quote
It makes a loss of over 41 thousand (not in travel costs) when the p****engers leave. It returns home with over 300 p****engers and makes a profit of over 22k when the p****engers leave. Have I got a setting in wrong in the config.tab or is this another bug?
(I ****ume that you don't refer to ST-experimental.)
pay_for_total_distance = 2 can lead to such a result.
Also, cost deduction at switching of the month might be involved.

Re: poor game flow in large maps

Reply #17 –
Quote from: whoami – on
As I wrote, my suggestions (if they matter at all) will not help (much) in the case of the p****enger/mail network.
Directly, it will only increase routing calculation effort to a smaller extent, because of the larger number of stations ("hops"). But usually, you will have more different lines, more interconnection points and more transfers, and that makes a big difference.

Do you mean that if I make multiple smaller Tram networks in my cities instead of one large network it would improve the flow of the game?

In Experimental the train made $721 profit on the outward journey with 3078 p****engers as apposed to a $41k loss in Standard.

Additional info re-train behaviour.

Train travels A-B-A.
From A to B
journey 1. 3078 p****engers, loss on arrival at B -$41k plus.
Journey 2. 3078 p****engers, loss on arrival at B -$42k plus.
Journey 3. 3078 p****engers, loss on arrival at B -$43k plus.
From B to A
journey 1. 325 P****engers, profit on arrival at A $22k plus.
Journey 2. 476 P****engers, profit on arrival at A $16k plus.
Journey 3. 124 p****engers, profit on arrival at A $4k plus.

This doesn't make sense and it doesn't matter if I set the pay_for_total_distance to 0, 1, or 2. I realise that these settings may not affect a current SVE game. But it doesn't detract from the nonsensical profit and loss that is being generated.
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #18 –
Quote from: Colin – on
Do you mean that if I make multiple smaller Tram networks in my cities instead of one large network it would improve the flow of the game?
More different lines means more schedules and transfers to check, so goods routing calculation will need more CPU time. From this aspect, a larger station coverage can support in reducing calculation effort.
The performance-wise effect of the station coverage setting depends, however, also on your network layout: The more tiles with houses are covered by several stations, the slower the game may become, because a generated p****enger will check more than one station for suitable connections. But the developers can provide a better, substantiated analysis on this complex problem.

As your savegame doesn't contain an extremely large network, there may be a performance bug. Additionally, Z9999 may be right when he said that ST has become "too heavy".

Regarding the obscure revenue: It's unrelated to the program performance problems, so could you please open a new topic for that, also providing the savegame with the train? After that, I will move my own replies and any remaining related text there.

Re: poor game flow in large maps

Reply #19 –
Quote from: whoami – on
As your savegame doesn't contain an extremely large network, there may be a performance bug. Additionally, Z9999 may be right when he said that ST has become "too heavy".

Oh dear! I thought my SVE contained a fairly m****ive network. I must be really naive.

Quote from: whoami – on
Regarding the obscure revenue: It's unrelated to the program performance problems, so could you please open a new topic for that, also providing the savegame with the train? After that, I will move my own replies and any remaining related text there.

Ok I'll upload my SVE with the latest additions and then open a new topic.
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #20 –
The station coverage is not important.

The problem in your savegame are not the size of the map but the shear amount of waiting persons. You have several stations with more than 300000 people waiting. Updating such a monster will of course take some time.

I will look into the code to make it smoother, although no promises ...

Re: poor game flow in large maps

Reply #21 –
Quote from: prissi – on
The station coverage is not important.

The problem in your savegame are not the size of the map but the shear amount of waiting persons. You have several stations with more than 300000 people waiting. Updating such a monster will of course take some time.

I will look into the code to make it smoother, although no promises ...

Thank You Prissi, I will look forward to any improvements you can make.

 In my next game (which may be awhile away due to this one being nowhere near completed) I will reduce the P****enger Level.
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #22 –
@ Prissi,

Does nightly 2640 contain your changes?
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #23 –
Hi Colin :)

According to the source code update history, Prissi's changes for speeding up re-routing have already been incorporated. So, does it improve the responsiveness of your game?

Knightly

Re: poor game flow in large maps

Reply #24 –
Quote from: Knightly – on
Hi Colin :)

According to the source code update history, Prissi's changes for speeding up re-routing have already been incorporated. So, does it improve the responsiveness of your game?

Knightly


No, if anything it's getting worse. I'm sorry to say.
I may not agree with what you say, but I will defend to the death your right to say it

Thought for the day

 When you are up to your backside in alligators, it is difficult to remind yourself that your initial objective was to drain the swamp.

Re: poor game flow in large maps

Reply #25 –
Quote from: Colin – on
No, if anything it's getting worse. I'm sorry to say.

Same here.
What we want is not to solve jumping convoi, but to solve unresponsive.
In recent simutrans, idle time is zero during long time even though convoi keep moving well.

Re: poor game flow in large maps

Reply #26 –
Quote
void spieler_t::step()
{
   if(  halt_list.get_count()>0  ) {
 
      uint32 it = halt_iterator_start;
      slist_iterator_tpl <halthandle_t> iter( halt_list );
      while(  it>0  &&  iter.next()  ) {
         it--;
      }
      if(  it>0  ) {
         halt_iterator_start = 0;
      }
      else {
         uint32 units_handled = 0;
         while(  units_handled<8192  ) {
            if(  !iter.next()  ) {
               halt_iterator_start = 0;
               break;
            }
            halt_iterator_start ++;
            // iterator until 8192 p****engers were handled
            units_handled += iter.get_current()->sum_all_waiting_goods();
            iter.get_current()->step();
         }
         INT_CHECK("simplay 156");
      }
   }
}

After looking at Prissi's changes, I think one of the reasons why the unresponsiveness problem has aggravated lies in the fact that >= 8192 ware (not ware packets) are re-routed in each step. For hub stations with large-sized ware packets (i.e. large menge), this will indeed help to prevent too much re-routing work in one step; but with non-hub halts, they usually have only small-sized ware packets (please see the attached image), so it needs to process many non-hub halts before the limit of 8192 can be reached.

IMHO, re-routing workload depends not on packet size (menge), but on packet count. More packets, more re-routing work, and vice versa. So, I would suggest to change from counting ware menge to counting the number of packets processed. Of course, the limit of 8192 also needs to be reduced accordingly.

Re: poor game flow in large maps

Reply #27 –
The problem in Colin save are that there are several stations with 300000 p****engers. Updating (rerouting) them takes nearly a second on my computer. During this no user interaction is possible, since such can be only done outside a step. The only other way would be that the stations themselves only handle 8192 each step, and indicate the result then. But this would require more complex logik, as changes of the p****enger numbers must be also taken into account (i.e. something like a dirty flag for ware in stations.) I will look into this next week.

Re: poor game flow in large maps

Reply #28 –
Edit : Post removed as the provided screenshot was wrong.
 

Re: poor game flow in large maps

Reply #29 –
Knightly,

I really only stumbled accross this thread randomly, but looking at your screenshot, I strongly suspect that you fell victim to a display bug: the sum of p****engers of the packages displayed is never 300k, not even close. I guess lots of destinations with letter >B are missing.
(previously known as "tttron")

Re: poor game flow in large maps

Reply #30 –
Display works only with integer coordintes with 16bit; therefore I think the scrollbar just messed up. It is very easy to time those stations and they have like 4000-30000 packets and cn take up to 800ms or rerouting (on this computer here).

Re: poor game flow in large maps

Reply #31 –
Prissi,

Really sorry! I didn't realise that the list is incomplete. I loaded Colin's save game again and looked at the same station, and the list kept growing gradually when I tried to scroll down.

Nevertheless, I think it is more logical to count the number of packets instead of the sum of menge. Hope you can consider this suggestion when you look at the code later.

Knightly


Re: poor game flow in large maps

Reply #32 –
[offtopic]
this:

Quote from: tttron – on
I strongly suspect that you fell victim to a display bug: the sum of p****engers of the packages displayed is never 300k, not even close. I guess lots of destinations with letter >B are missing.

should be fixed in revision 2642.
[/offtopic]
Parsley, sage, rosemary, and maggikraut.

Re: poor game flow in large maps

Reply #33 –
In the current implementation, if the goods of a station are rerouted, simutrans does a BFS for each packet. Since no heuristic is used, it is possible to do this in only one BFS, until the target for each packet is found.

I will try to implement this, if there is any free time next week...

Re: poor game flow in large maps

Reply #34 –
Please check again with curretn nightly. Should be much smoother (only lagging on my eeePC).