[ale] "per-process virtual address space" limit only for x86 architecture?

[Michael B. Trausch]

On Mon, 2008-07-28 at 08:34 -0400, Jerry Yu wrote:
> Only for x86, the RHEL comparison chart lists limits on per-process
> virtual address space.  What about x86_64, PPC, and other
> architecture?
> URL at http://www.redhat.com/rhel/compare/
> exact verbage is "Maximum x86 per-process virtual address space"

This is slightly misleading the way that they have it.

The summary:  The kernel uses 1 GiB of space out of the virtual address
space given to any given process on x86 and x86-64 systems.  On x86
systems, the virtual address space is 4 GiB.  On x86-64 systems, the
virtual address space permitted today is 48 bits, growable to 64 with
newer CPUs, but the Linux kernel only gives 47 bits (128 TiB) of address
space to a process on an x86-64 system (at least on mine; Ubuntu Hardy
with a 2.6.24 kernel).  Mostly, the per-process virtual address space on
any 64-bit system is going to be larger than you'll need to worry about
for all but the very most large of applications.

The full story, AFAIK:  First off, the address space that a program
receives on an IA-32 system without 64-bit capability (or a 64-bit
system running a 32-bit kernel) is 4 GiB.  This is the maximum supported
for any process, because that is the maximum supported by the
specifications for IA-32 without PAE.  Even with PAE, an application is
constrained to a 4 GiB address space because PAE only enables the CPU to
support more than 4 GiB of physical RAM.  PAE enables 36-bits of
physical memory addressing, while applications still follow a 32-bit
model for virtual memory addressing.

On AMD64 (x86-64) architecture CPUs, there is a theoretical limit of 16
EiB (2^64 bytes) for an address space.  Currently, only 48 bits are
enabled on most CPUs, which gives an address space of 2^48 bytes, or 256
TiB.  It is interesting to note that the address space given to any
process on an x86-64 system is vastly larger than the maximum amount of
physical memory that today's x86-64 CPUs can address, though this is not
unlike the 386 when it came out.  The strange part is in the way that it
is allocated:  Present 48-bit implementations have a map as follows:

  0x0000000000000000 - 0x00007fffffffffff = Lower half of address space
  0x0000800000000000 - 0xffff7fffffffffff = Unused
  0xffff800000000000 - 0xffffffffffffffff = Higher half of address space

What I am not sure of is the exact details surrounding the memory that a
process thinks is available for its virtual memory space.  I modified
the program used by an article on memory addressing in Linux [1] to try
to run out of memory quicker on a 64-bit system (that is what I run) to
see what I could find.

The program was able to successfully allocate 131,071 GiB of address
space, and failed when attempting to allocate an additional GiB, when I
modified it to allocate one gig at a time instead of one meg at a time
(allocating one meg at a time would've been bad; it was heading for a
real OOM condition in the amount of metadata it was tracking before too
terribly long).

To duplicate the result, I modified the amount of allocated memory to 2
GB minus one byte (the maximum allocatable amount without overflowing
the integer parameter to malloc(3)) and ran it again.  It was unable to
allocate 131,072 GiB, but it was able to allocate 131,070 GiB.  What I
gather from this experiment is that the Linux kernel still uses the
upper 1 GiB of memory space, and that a process has 131,072 GiB of
address space (a total of 128 TiB).  This is 50% of the present
theoretical limit of 2^48 (256 TiB) bits of address space, so the kernel
doesn't give a process all that it could.

I don't know about IA-64, or PPC64 systems.  I would imagine that if
they use 64-bit virtual memory systems than the address space would be
vastly larger, depending on the operating system.  I cannot imagine that
Linux on any 64-bit architecture with virtual memory is going to give
less than it does on x86-64 systems, but I could be wrong.  If I had
such hardware, I could experiment and see what happens.  Maybe someone
else on this list does?

	--- Mike

[1] http://preview.tinyurl.com/wvxwp
-- 
My sigfile ran away and is on hiatus.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.ale.org/pipermail/ale/attachments/20080728/f8fa50b8/attachment.bin