Reducing lag from membrane updates

I’ve been trying to look into what makes very big cells as laggy as they are. One thing that I came across was organelle splitting, which can sometimes cause noticeable lag spikes as you know. Here’s an issue about it. However, from my testing it seems that those lag spikes are caused by membrane updates instead of the organelle splitting itself. Every organelle split marks the membrane as “dirty”, which leads to the membrane being updated shortly after. The membrane update can take a lot longer than the organelle split itself. Here’s some data I gathered playing through Godot on a non-gaming laptop:

Data

SplitOrganelle takes 0.5-15ms. The time taken was pretty random but generally scaled with size.

Membrane updates:

  • Primum thrivium : 17ms

  • random enemy cell in free build: 70ms (small lag spike)

  • eukaryote with 20 organelles: 120ms (a bit bigger lag spike)

  • big cell with 65 organelles: 450ms (big lag spike)

  • from cache: 0.1ms

Benchmarks

Benchmark results for MicrobeBenchmark v1

Stationary microbes score: 5.255

AI microbes score: 8.627

Spawns until no 60 FPS: 43

Microbe stress average FPS: 54.665

Microbe stress min FPS: 39

Alive microbes: 43

Waiting for microbes to die: 48.74

Microbe deaths minimum FPS: 36

Remaining microbes: 34

Total test duration: 138.2

CPU: Intel(R) Core™ i5-8250U CPU @ 1.60GHz (used tasks: 4)

GPU: NVIDIA GeForce MX150/PCIe/SSE2

OS: Windows

How I collected the data

I couldn’t get normal profiling to work (I’m on Windows and use Visual Studio) so I did the following instead: I measured game ticks before and after the wanted method and then printed the difference to the console. I then converted the ticks (microseconds) to milliseconds. I measured the ticks with Time.GetTicksUsec().

var time1 = Time.GetTicksUsec();
MethodYouWantToMeasure();
var time2 = Time.GetTicksUsec();
GD.Print(time2 - time1);

So what are some takeaways from the data? While SplitOrganelle can take some time, it doesn’t cause the lag spikes. The lag spikes are caused by membrane updates. The more organelles a cell has, the worse the lag spikes are. Membrane updates scaled consistently with organelle count. Updating to a cached membrane is really fast luckily.

So if the membrane update is the culprit, is there something that can be done about it? As it turns out, yes!

I did some tinkering and made my own algorithm that updates the membrane in under a millisecond for small cells and a few milliseconds for big cells. That is up to about 100 times faster. It completely eliminated growth related lag spikes with the aforementioned cells for me. To get lag spikes from organelle splitting now, I’d have to make a cell with at least hundreds of organelles. At that gargantuan scale, it seems that SplitOrganelle becomes the new bottleneck.

The algorithm works a bit differently from the old one, so it also makes slightly different looking membranes. I tried to keep the end result similar enough. The membranes have more points in them making them smoother. The basic membranes have some waviness to them, the cell walls don’t. Compare these pictures:

Membranes


before


after


before


after (circle)


before


after


before


after


before


after


before


after

The algorithm can also better wrap around organelles, mostly fixing issue 4117. It does that by giving the membrane more distance from organelles if the cell is a eukaryote, which you can kind of see in the “after” pictures. That worked overall better for some reason than giving the membrane hex positions instead of organelle positions.

The algorithm also fixes issue 1274 by making the starting square much larger.

Fixed issue 1274


Before (the cell is a line of 100 Cytoplasm, just invisible because the membrane generated wrong)


After (same cell, no problem)

This is kind of unrelated, but I also noticed that there’s a visible seam in cell walls. I think you know about that, but I couldn’t find a Github issue about it. It can actually be fixed by changing one line of code. If that line is removed, the seam disappears, but that also makes the cell wall textures denser.

Seam


Before (look closely to see the seam in the upper left corner)


Before (notice normal texture density)


After (notice no seam)


After (notice denser texture density)

TL;DR: Membrane updates cause lag spikes. My algorithm gets rid of them. It also fixes a couple other bugs.

I could make a PR for this reworked algorithm if you would like. It would take me a little time to get everything cleaned up for that, and the PR would probably require testing from others to make sure the membrane looks good on a variety of cells. A few questions first:

  • Should I make the PR?
  • Do the membranes look good?
  • If they don’t, what should I change?
  • Should I add the cell wall seam fix in?
8 Likes

Yes, definitely. Even if it never gets finished, someone might be able to do future work based on it that could then be finished.

I think they look good enough (maybe a bit more blobbiness would be good to preserve though to have a more natural, less uniform look?).
Though changes to how things look that people are used to will be met with extra resistance.

I think they could try to be more blobby like before to have a more natural look instead of a more round and uniform look of manufactured things.

I think about the only person who might object is Untrustedlife but they are quite unlikely to comment on this. I don’t think anyone else knows why that wall generation code has the changes it does compared to the normal code.

Thanks for the feedback. By the way, I noticed I missed the mesh building part of the code in my data collection. That often adds some milliseconds to the membrane update. I couldn’t find any way to make that faster. So the total membrane update time is usually about 5-15ms for my example cells. That’s still a big improvement for medium and big cells.

I’ll have a look if I can tweak the membranes to look a bit more natural and more similar to how they used to look. Then I’m gonna clean up the code and make a PR for this.

1 Like

I doubt there’s really any way to speed that up other than trying a different approach to sending the data to Godot, which might need a separate C++ module. That has been talked about before (rewriting some performance critical parts of the game in C++ that can be decoupled enough from the other code to make that possible).

There’s some alternative approaches I’ve thought about but would need a bit of a change to the approach:

  • Pregenerate the default layouts for species when exiting the editor (and ensure the caching time is long enough to keep them in memory). As an addition membranes for some organelle growth could also be precomputed, but this’ll quickly take exponential time and not all of the membrane stages will end up being used in the end
  • Cache the mesh instances as well so that when a mesh is needed from the data, a cached version could be used if cells with the exact membrane shape are currently in the game

Sounds good.

Would you mind sharing more about that algorithm? This is a beast I started fighting a bit ago, and I knew it was going to need an algo swap. I’m wondering what you came up with if you want to name it or do a quick explanation. But yes, 100% make a pr.

As for making the mesh generation faster, we honestly just need to swap that algorithm out as well. Nunz (Our main graphics guy) recommended Dual Contouring.

Oh also, after that pr, want to submit a dev team application? :slight_smile: We love getting all the help we can.

1 Like

Unless I’m misremembering, the conversion of the generated membrane points to a mesh, literally does a few floating point multiplies and just directly creates a couple of vertices for each point. I don’t think there’s any chance of making that any faster whatsoever. Only by directly getting write access to the data buffers to be uploaded to the GPU could that be faster. Well I guess due to access checking in C#, just porting the code to C++ might give 30% speed increase, but that would be it.

Basicly, it starts with 40 points along the edges of a big square with the cell in the center. Then each point finds its closest organelle and moves close to it in a single step. Then it makes a second buffer that goes around those points and places new points evenly. Then the new points are shifted using sine waves to make the membrane wavier.

It’s probably easier too see from the code once I make the PR. You can then ask any questions about the details.

Thanks for the suggestion, I’ll think about that! One concern I have is that I might not be very active in the near future. My activity tends come in bursts based on when I have time and inspiration. Is that fine or should I wait until I’m more ready to be active?

You checking in every now and then is a lot better than a large portion of the people accepted to the team. A lot of people just disappear after doing a little bit initially, and then never return.

Done right, I think it might be possible to replace the points part of the membrane generation code, and just directly make the mesh from sampling the above algo. While the mesh generation itself might be slower, It optimizes the above algo further, and provides a crispier result with a dynamic point count.

There’s now an open PR about this:

1 Like