Checksum of code segment

[elminster]

As like all online games, the one I'm working on has got hackers. We already have a load of normal checks, various methods of detecting various cheating tools, anti-debugging techniques, amongst other things. But we found one of the recent hacks is patching our code.
From what I read, doing a checksum of code (combined with guard pages) is a good place to start to detect it. I've read quite a few references, papers, articles, etc but barely any actually list the code I'd be after. I had a quick go at doing it myself (Just to get something working) and so far I got:

Code: Select all

   HINSTANCE hInstance = GetModuleHandle(NULL);
   PIMAGE_DOS_HEADER dosHeader = (PIMAGE_DOS_HEADER)hInstance;
   PIMAGE_NT_HEADERS ntHeaders = (PIMAGE_NT_HEADERS)((DWORD)hInstance + dosHeader->e_lfanew);
   PIMAGE_SECTION_HEADER sectionHeader = IMAGE_FIRST_SECTION (ntHeaders), codeSection = NULL;

   for (int section = 0; section < ntHeaders->FileHeader.NumberOfSections; ++section)
   {
      if( sectionHeader[section].VirtualAddress <= ntHeaders->OptionalHeader.AddressOfEntryPoint //Entry point is within section (i.e. Code segment)
         && ntHeaders->OptionalHeader.AddressOfEntryPoint < sectionHeader[section].VirtualAddress + sectionHeader[section].Misc.VirtualSize )
      {
         codeSection = sectionHeader;
         break;
      }
   }

   if(!codeSection) return 0;
    //uses common crc32 implimentation
   return crc32buf((char*)((DWORD)hInstance + codeSection->VirtualAddress), codeSection->Misc.VirtualSize);

I know I don't need to checksum the whole program code, but done it for simplicity so far. Also I seen several checks done to identify the right segment (check for ".text"/"CODE"/etc, check for Code+read+Execute flags, and some others), but this also works.
I've tested it in various stages of the game and it seems to be consistent, but when compiled with /DYNAMICBASE (MSVC extention for Address Space Layout Randomization) it differs. I'm not entirely sure if that actually randomizes elements of the code segment or splits it up, or something else, but detailed documentation of implementation on windows isn't easy to come by (There's several papers about defeating it, might be able to infer it from them). Obviously it's an advantage to have it enabled for security.

Although maybe I've done it wrong, maybe I'm going about it the wrong way, I'm not sure since I've only read theory on it and only small bits of code. We already use a executable packer at the moment.

I'm wondering if anyone has already done something similar or has a better idea of what I should be looking at?

[EvanED]

I can imagine there could be pointers which move around with ALSR and hence the checksum differs.

Would it be satisfactory to just checksum the EXE file and any DLLs it uses? (And possibly other files -- there are plenty of hacks that don't require changing code at all, just stuff like textures.)

[korona]

I would argue that if an online game can be hacked by making changes to the client then the client/server design is flawed.

[EvanED]

I would argue that if an online game can be hacked by making changes to the client then the client/server design is flawed.

Aimbots, just to name one counterexample?

[sourmìlk]

That's basically the only counterexample. The client should only have access to the functions that the player has access to. There's not much of a way of ensuring that it's actually the player performing those functions, but you shouldn't be able to do anything client-side to circumvent it. No data sent to the server should allow the player to do something he couldn't have done already.

[elminster]

Yeah we already know about this. Rule #1 of online games is treat everything the client does as a potential hack.
They're not doing anything that breaks the mechanics per-se, it's mostly just graphical edits to help them (e.g. highlighting enemy players). Also there is a design flaw in the fact that there's no latency compensation, which means that particular scenarios can be tweaked enough (sub second timings)that it would give you an edge, but it's difficult to accurately determine. The hacks they made really wouldn't give you any notable advantage in most cases (in very balanced ones it could give an edge though).
Also, someone did actually make an aimbot before, but aiming is only a small part of the game really... so it wasn't hugely useful. The guy who made it sent us a copy when he stopped playing.

Anyway, the point of this was to reduce the likelihood of any other hacks coming up. I was hoping to avoid many more specific targets (e.g. Preventing Cheat engine, SoftICE, Ollydbg, IDA Pro, unpackers, etc) and do more blanket protections.

[mbrigdan]

I'm pretty sure that if someone is willing to put in the effort, then Cheat Engine can make use of Kernel calls. I don't know if that's the kind of thing that's even preventable with normal measures.
Perhaps doing some sort of checksum of the memory that the game uses, and noting if its been changed when there hasn't been a call that should have changed it?

[elminster]

I'm pretty sure that if someone is willing to put in the effort, then Cheat Engine can make use of Kernel calls. I don't know if that's the kind of thing that's even preventable with normal measures.
Perhaps doing some sort of checksum of the memory that the game uses, and noting if its been changed when there hasn't been a call that should have changed it?

Well yes, that's exactly what I was explaining I wanted to do, but ASLR was screwing up the checksums.

In a rather lucky find, someone actually wrote a Masters paper on calculating the checksum of a code segment loaded in memory with ASLR... I shit you not, someone actually wrote a paper on EXACTLY what I was after.
Essentially he says that ASLR modifies all the addresses (e.g. mov eax, [address] -> mov eax, [address + offset]) and he does the checksum by filtering out any address that's been relocated and getting the checksum of the remaining parts. Although that could still be broken... but good enough for now.

Don't know how they allowed him to cite Wikipedia, and really... that doesn't look like a masters level project. Doesn't take much to figure out you can checksum everything but modified addresses; hell, I only looked into it for a few days really.

Would it be satisfactory to just checksum the EXE file and any DLLs it uses? (And possibly other files -- there are plenty of hacks that don't require changing code at all, just stuff like textures.)

We already checksum files before loading and validate they've not changed since loading (currently moderately hard to break, but good enough for now). The issue is more to do with modifying code in memory.

edit: Also there seems to be some patent on a method of "Code checksums for relocatable code"
Wouldn't have guessed this stuff would be this complicated.

[TNorthover]

We already checksum files before loading and validate they've not changed since loading (currently moderately hard to break, but good enough for now). The issue is more to do with modifying code in memory.

Quis custodiet ipsos custodies.

[mbrigdan]

I'm pretty sure that if someone is willing to put in the effort, then Cheat Engine can make use of Kernel calls. I don't know if that's the kind of thing that's even preventable with normal measures.
Perhaps doing some sort of checksum of the memory that the game uses, and noting if its been changed when there hasn't been a call that should have changed it?

Well yes, that's exactly what I was explaining I wanted to do, but ASLR was screwing up the checksums.

I'm not too familiar with ASLR, so I could be totally off base with this, but my idea wasn't to compare the code with a fixed checksum in the application, but to compare it against an "initial" checksum, presumably made after the code has been loaded into memory and ALSR has done its work. Of course, this would only work if ASLR only happens at the very beginning, and nothing is randomized after you take that first checksum.

Of course, as TNorthover points out, that still won't stop someone from modifying the checksum code itself to be non-functional, and then going to town on the rest of the program.

Spoiler for getting off topic:

Spoiler:

Which, really, is the problem with having an untrusted client. No matter what steps you take, you can never be sure that someone hasn't reversed your network protocols and written their own client, free of any restrictions you try to create