[ale] Can someone help me understand memory usage?

[Jeff Hubbs]

Geoffrey wrote:

> Jeff Hubbs wrote:
> 
> 
>> Yeah, but 2.4 kernels are "greedier" than 2.2 and before.  As I 
>> understand it, you can no longer view gobs and gobs of free RAM as a 
>> sign of system health.  Rather, the OS stakes out big swaths of 
>> otherwise unused RAM and hangs onto it.  And, eventually, it will 
>> start filling up swap.  However, this is not so much a sign of 
>> impending meltdown as the kernel going, "hm, here's a section of RAM 
>> that hasn't been used in hours; I'm throwing it to the swap file."
> 
> 
> I don't know how 2.4 kernels have changed, but either I don't understand 
> what you've just said, or I think it's a poor direction to head.  The 
> LAST thing the OS should do is use swap.  It should look for unused 
> memory and take advantage of it.  Using swap is kinda like you've got a 
> bus load of folks and their's no more room, so you take the bicycle off 
> the roof rack and hand it to the next guy trying to get on the bus. AND, 
> the bus then stays behind the guy on the bike.  Point being, using swap 
> slows down everything.
> 
> Did I misunderstand your statement, or are you saying the OS will use up 
> memory quickly because of the allocated chunk sizes, therefore hit the 
> swap much sooner than it used to???
> 
> 

Well, you're kind of pushing me to the wall as regards my knowledge and 
explanatory power...but here goes! :-)

It's as though 2.4 kernels have done away with this notion that you 
should have as much free RAM as possible at any given moment.  Instead, 
the kernel is eating into free RAM not because it needs to but because 
it wants to.  Perhaps the impetus here is one of planning ahead:  The 
kernel needn't take steps to grab more memory when it suddenly needs it 
IF it's already grabbed it.  In other words, VM OS memory management 
seems to have historically been predicated on the fear that someone was 
going to steal RAM out of your machine at any moment.  Modern Linux 
kernels seem to act like, if you gave it half a gig, it's gonna act as 
though it's going to stay in the box.  It also seems to act as though it 
does not harm you, the user, if it has taken up a total 426 of 512MB 
even if it's not really doing anything (which it doesn't).

I probably said the wrong thing when I said "it will start filling up 
swap;" I probably should have said "using some" instead of "filling up." 
  My current intuition is still being informed as I work with 2.4 
kernels, but the sense that I've gotten so far is that the kernel will 
determine that, of the RAM that is actually being NEEDED (as opposed to 
allocated) at any given time, it will find stuff in RAM that just 
doesn't get accessed much (mind you, this really only kicks in after 
some run time passes and some significant activity occurs) and it will 
get thrown to the swap file.  And there it will sit, not harming you nor 
harming performance.  If te kernel has to go to swap to get it, well, 
fine, that's ONE access that you won't exactly need a bathroom break 
for.  However, the kernel will have changed its mind about what's needed 
right away and what's not and therefore it's not going to have to repeat 
the access the next time.  My modern intuition says NONZERO SWAP VOLUME 
IS NOT EVIL.  Having to go back to it frequently and protractedly IS 
evil.  What I'm trying to say is that under light to medium-light 
loading, swap volume will come up off the zero peg but it's no big deal 
- it's when the disk I/O to swap becomes significant that you have a big 
deal.

Under VMS, on a production system that loads up in the morning and gets 
run real head most of the day, there would be a point early in the day 
where the box kind of made an "Urrrp..." because it suddenly decided to 
begin making decisions about VM management and began busily arranging 
its swap.  I think what has happened under Linux is that they've tried 
to "soften the knee" in the performance curve.by letting it make those 
desicions pretty much all the time.  I'd surmise, then, that Linux boxen 
can handle getting heavily loaded up with a bit more grace and charm.

- Jeff


---
This message has been sent through the ALE general discussion list.
See http://www.ale.org/mailing-lists.shtml for more info. Problems should be 
sent to listmaster at ale dot org.