Friday, November 23, 2012

Musings On the Nature of Gamebryo, #1656E

For literal years now (since mid-2009), I've tried to tell people that Gamebryo's -- specifically Fallout 3's (and later Fallout NV's; which is hands-down the greatest expansion pack ever) -- reliability is inversely proportional to the number of followers you have in tow.

Over the past six or so weeks since Skyrim got boring and my attention flitted back into FNV territory, this has been demonstrated yet again.

In my first "hardcore-hardcore" game -- no companions, no freebie equipment; all by my lonesome, using only armor and weapons that I could find, steal, buy, or loot -- my FNV set a personal record for longest continuous up time without a crash: about four hours and ten minutes -- and at the end of that, it didn't crash; I exited normally because... y'know... it had been four fucking hours and I really needed to get up and move around before I got a goddamned blood clot or something.

Shortly thereafter, Mystery-chan used her infernal powers to cajole/guilt me into letting her come along so we could adventure together.

This brought up-time back into more familiar -- but still outstanding by my standards -- territory of about three to three and a half hours play before a crash; depending on what we were doing.

Since then, I've started other games and characters to test other things without messing up the hardcore game; or what-have-you.

With a second companion in tow, average up time falls back to my old normal three hours -- sometimes as light as two and a half if I'm running a mod I know damned well I shouldn't because of its shoddy coding (we won't go into which mod or why -- it isn't that kind of blog).

With all three of my beloved girls in the party, we average about two forty-five; and it usually ends up being a crash on loading a new cell.

Two forty-five to three hours, I should note, is a comfortable zone for me. It's not great reliability, no; but it's about the area where I start needing to get up off my dead ass and move around.

Last night, I got in a couple play sessions with all three dogs in tow. While they're not following me; they are none the less companions. They follow their respective mistresses, have companionscripts, and all that. The only real difference is the target of the following packages. The companionscripts for the dogs are comparatively simple -- 154 lines each (less than a quarter the size of the girls' scripts) -- but nonetheless, six "companions" on screen at any given time had an immediate and easily noticeable effect.

Day before yesterday, when I set the game up and did some early-game crap around Goodsprings, I played for an hour and thirty minutes before the crash.

Last night/this morning, according to the date stamps on the save files, I played for precisely one hour and forty minutes before it crashed.

Even while running, though, it was causing problems. Framerates were noticeably lower; and the engine's script bottleneck was in full and blatantly obvious effect -- it actually stopped Mystery-chan and Maeva's hardcore code from functioning correctly.

This, I should note here, is one of the reasons I haven't jumped in and released a companion for you monkeys to break that features the full hardcore companion mode. Certain mods that take over the NPC's AI can cause the code to break; and if your game's cache is overloaded or nearing full, the code can stop functioning correctly at random -- for instance, last night the code would run correctly if they had gecko steaks in their inventory to eat; but wouldn't run if they had yum-yum deviled eggs. Why? Who the hell knows. Gnostic mysticism is easier to make sense of than this game. I've tried scripting out around it several times; but when the hardcore code fails it also breaks their following behavior -- which then has to be manually reset... it's kind of a mess and I haven't been able to make it suitably reliable yet unless kept to one companion -- and we all remember how well that recommendation went back in RR.


I've noticed time and again that the two biggest culprits in system load (and therefore crashing) are loading new cells -- especially in the open wasteland -- and at the start of combat.

"But Nos!" I can hear you hunt-pecking out on your keyboard; "New cells don't load in the wasteland! Cells only load when you see a loading screen!"

You're wrong. The wasteland is actually broken up into cells; a number of which adjacent to the cell you're in are displayed in the distance (controlled by the Ugridstoload setting in your .ini). When you come within appropriate range, the next cell in the distance begins to load; LOD at first, but as you move to within range as defined by your graphics settings, full objects will "pop in". The moment of pop-in is where I find most of my crashes occur by a wide margin.

A close second is when the combat AI fires up. As I've said in the past I don't know what in the unholy Hell the AI is doing when it does that... but the system load spikes and sits there for several seconds. I don't know whether it's just switching on combat styles; or if it's plotting specific behavior... hell, it could be plotting to overthrow humanity and using FNV as a simulation to test battleplans for all I know.

It seems once again like I've hit the wall; and FNV just doesn't have the ability to handle what I want it to be.

I'd hoped Skyrim with its "new and totally original" (excuse the noise I'm making; I'm totally not snickering...) engine would be a step in the right direction. Sadly Papyrus is utter shit; and their improved script queuing overloads faster and easier than it did in Gamebryo.

So yeah. Maybe it's just the four hours of sleep talking, but I'm pretty discouraged tonight. I may try a reduced party with one companion and one pooch... but beyond that I think the dogs will need to spend most of their time lounging around the estate like I originally intended.


  1. And now for another of Herculine's only partially-relevant comments:

    It is indeed amazing how the more they "improve" Gamebryo the more it also stays the same. As I've mentioned in my own blog, I've been on a bit of a Morrowind bender lately. However, it has been more troubleshooting getting the game to run properly with 200+ mods and graphic-enhancing programs installed than it has been actual gameplay; today I started the game over from character generation for the fourth or fifth time since I got back into the game a couple of weeks ago. I was actually starting to tell myself that if I'm ever going to learn any of this companion scripting stuff, basic Morrowind companions might be the best place for me to start. Then today I encountered a recurring issue where, after I've played a certain length of time, activated a certain number of quests and affected a certain number of cells, suddenly the game will unavoidably and immediately crash upon trying to load any game save made after that certain point. With the help of using Wrye Mash to repair the save file and analyze the changes made in turn, I discovered the source of the problem: a single companion character. Now that I've abandoned that specific character and started using different companions, suddenly everything is fine again and suddenly I'm not feeling so inclined to learn companion scripting after all.

    1. "I discovered the source of the problem: a single companion character."

      I believe it. I don't know whether I've ever recounted the story before, but just after I got the Estate working and livable, I was redoing the girls' "stay here" dialogue to include a new option that would only apply when "at home" -- so that they'd sandbox normally anywhere else, but at home would swap their armor for loungewear; remove backpacks and holsters and so on.

      Two of the worked perfectly. Third never worked. I'd use it, and she would play the dialogue; inventory would flash like it was changing, but then go back to normal armor and she'd keep following. Must've reset the code a dozen or more times; never worked once.

      Popped into FNVEdit (no, you don't want to know how much of a pain it is to identify a specific line of dialogue in TESSnip), deleted the one dialogue response.

      Went back into the GECK, recreated a new one with the same name; same dialogue; same script attached. Worked perfectly ever since.

      Some days, there just is no rhyme or reason -- stuff gets corrupted somehow in the plugin itself, and no matter how the GECK displays it or how many times you redo it; there's no solution except to trash the offending record(s) and create it anew.

  2. It sounds to me like it's running out of memory. If you had a 64bit system with more than 4GB of memory, the 4GB LBA patch would probably help (or at least defer the crash).
    Assuming you aren't already using it that is.

    As for that script weirdness, it's probably yet another GECK bug.

    1. "If you had a 64bit system with more than 4GB of memory, the 4GB LBA patch would probably help (or at least defer the crash). "

      Yes but if I had a 64 bit system with eighty-seven gigs of RAM most games wouldn't work any better (or right at all) without the large-aware patches that are hit and miss. I know Skyrim's patch has been invalidated two or three times now by new game versions.

      (I have a 64 bit processor but only 32 bit Windows because whoever owns eMachines is a cheapass; and I'm too much of a cheapass to buy a whole new version of Windows)

      "It sounds to me like it's running out of memory."

      Fair call, and I won't argue the point since I agree; but what gets me is that it isn't system-wide.

      I don't do it often since I have to be in a particular mood; but awhile back I played a game of Sins of a Solar Empire straight through -- to the tune of seven hours and change. No crash; no audio bugs; no significant slowdowns. When the game was done, I exited and after the computer finished clearing its cache I went back to using it normally. Other games are the same way.

      Fallout NV though is troublesome. Sometimes it can be "fixed" by clearing the cache and restarting the game; sometimes the memory leak makes a full system restart necessary. Sadly it also has a habit of killing my video drivers (requiring restart).

      What gets me is only two things do this to my system: Gamebryo titles (getting progressively worse as you move through the successive titles) and Flash's Firefox plugin -- which has a memory leak big enough to drain the Great lakes. If I don't visit flash-enabled sites, and don't touch any Gamebryo games (or their toolsets) the system can have a week plus of up-time without a hitch one.

      "As for that script weirdness, it's probably yet another GECK bug."

      Far as I know, it's bottlenecking -- the game only gets allocated so much CPU time; and only a certain percentage of that goes to script processing. When it fills up, calls keep getting sent but are ignored and no more will run. The larger and more complex scripts are (and my personal companion system are pretty heavy duty) the faster you hit that saturation point.

      I've tested out that simpler scripts let you run more iterations (more companions) at once before the bottleneck; but at the same time people with computers far more powerful than my Bread Toaster MKII here have had massive problems with the scripts at similar ceilings to what I've seen; so I'm not willing to call it a simple "buy a better CPU" issue. There's something else going on in there; but sadly it's something we can't directly observe, quantify, or affect so as to attempt improvements.

    2. The 64bit thing helps because Beths Gamebryo really sucks at managing memory and more gives it more room to sprawl out. Part of the Oblivion Stutter Remover actually replaces the memory system to help control this, though I don't know if the fallout versions work.

      As for the scripts, I suspect what's happening is that the engine is using the common technique of allocating a fixed amount of cpu time to the scripts to stop a rogue script locking up the game. It's just really bad at adapting to faster cpus.