First, we'll discuss the 4GB RAM limit on x86 systems. The 4GB limit is hardware-based (well, almost - I'll explain the workaround later that allows up to 128GB) - the x86 architecture can really only address memory addresses at 2^32, or 4,294,967,296 (4,294,967,296/1024x1024=4096, or 4GB). So, no executable code can be at any address higher than the 4GB "boundary". However, there were 2 workarounds (hacks) that Intel put in place awhile ago to try and extend the life of the 32bit platform until it could get it's 64bit Itanium platform mainstream, and these were PAE and PSE (read here
for more on why Microsoft chose to implement PAE rather than PSE).
The hack you speak of, used in Windows, is called Physical Address Extension (PAE), which was added by Intel in the Intel Pentium Pro and Pentium II lines of CPUs (and continued subsequently to this day) to allow data (and data ONLY) to be mapped into RAM above the 4GB boundary - however, there is a set of limitations on this window. The first limitation is that this window to the RAM above 4GB *must* be mapped into a contiguous memory address range in the application's original 2GB (or 3GB, with /3GB) process space, because again the x86 architecture can only address memory up to the 4GB range (so mapping it into this window allows it to be addressed, with the application handling the management overhead). That brings us to the second limitation, specifically that this window must be managed entirely by the app, as I mentioned previously - there is no using the OS's memory manager, as the OS does not know how to handle memory above the 4GB boundary, thus placing an overhead on the app. The last limitation is that the memory window *must* not contain executable code (again, the CPU's eip register doesn't understand addresses above 0xFFFFFFFF, the address limit of the x86 architecture).
It's a hack, and if you want to actually *really* use memory above 4GB, you *must* use a CPU architecture that supports it (Intel's Itanium ia64 or the AMD/Intel x86-64, or x64).
The section portion of this sticky is the problem people see when they boot up an x86 system with 4GB of RAM, but Windows only reports the available RAM as somewhere between 3GB and 4GB. Rather than be lengthy about it and rehash old threads, I'll simply point to them to explain why this happens:
1. Start here
for a technical overview of how this can happen, and
2. Read this
, and this
to see real-life examples of this happening, and what can (and cannot) be done to resolve it.
Now it's been stickied, so if you ask these types of questions going forward, you will simply be pointed to the contents of this thread. These are not Windows problems, these are architectural limitations of the 32bit architecture - especially the second issue, where the BIOS reserves memory for hot-add/hot-swap memory, PCI-X/PCI-E buses, and (if it applies) your 256MB or 512MB video card reserve memory resources for their usage, causing the discrepancy in reporting of installed memory vs *available* memory to the OS.