Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Tue, 26 Jun 2001 15:58:38 -0400
From: Jason McMullan <jmcmul...@linuxcare.com>
To: linux-ker...@vger.kernel.org
Subject: VM Requirement Document - v0.0
Original-Message-ID: <20010626155838.A23098@jmcmullan.resilience.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.15i
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Tue, 26 Jun 2001 20:18:16 GMT
Message-ID: <fa.l9ipguv.b249q3@ifi.uio.no>
Lines: 144
Here's my first pass at a VM requirements document,
for the embedded, desktop, and server cases. At the end is
a summary of general rules that should take care of all of
these cases.
Bandwidth Descriptions:
immediate: RAM, on-chip cache, etc.
fast: Flash reads, ROMs, etc.
medium: Hard drives, CD-ROMs, 100Mb ethernet, etc.
slow: Flash writes, floppy disks, CD-WR burners
packeted: Reads/write should be in as large a packet as possible
Embedded Case
-------------
Overview
--------
In the embedded case, the primary VM motiviation is to
use as _little_ caching of the filesystem for reads as
possible because (a) reads are very fast and (b) we don't
have any swap. However, we want to cache _writes_ as hard
as possible, because Flash is slow, and prone to wear.
Machine Description
------------------
RAM: 4-64Mb (reads: immediate, writes: immediate)
Flash: 4-128Mb (reads: fast, writes: slow, packeted)
CDROM: 640-800Mb (reads: medium)
Swap: 0Mb
Motiviations
------------
* Don't write to the (slow,packeted) devices until
you need to free up memory for processes.
* Never cache reads from immediate/fast devices.
Desktop Case
------------
Overview
--------
On the desktop, interactivity is king. We don't want to eat
lots of I/O bandwidth paging in and out, however we also want
to cache as much of the FS as possible, to speed compiles and
multiple operations over the same sets of files.
Balancing this is the notion of 'cache-hit-rates'. If our
access patterns aren't hitting cache, but disk instead, don't
swap out processes, just shrink the cache. Contrawise, if we
have good cache hit rates, swap out the idle tasks.
Machine Description
-------------------
RAM: 32Mb-1Gb (reads: immediate, writes: immediate)
HD: 1Gb-100Gb (reads: medium, writes: medium)
CDROM: 640-800Mb (reads: medium)
DVD: 1Gb-8Gb (reads: medium)
Swap: RAM size (HD speeds)
Motivations
-----------
* If we're getting low cache hit rates, don't flush
processes to swap.
* If we're getting good cache hit rates, flush old, idle
processes to swap.
Laptop Case
-----------
Overview
--------
Same as a desktop, except now you must treat the HDs as
packetized devices for power-saving.
Machine Description
-------------------
RAM: 32Mb-1Gb (reads: immediate, writes: immediate)
HD: 1Gb-100Gb (reads: medium,packeted, writes: medium,packeted)
CDROM: 640-800Mb (reads: medium)
DVD: 1Gb-8Gb (reads: medium)
Swap: RAM size (HD speeds)
Motivations
-----------
* Keep packetized devices as continuously-idle as possible.
Small chunks of idleness don't count. You want to have
maximal stetches of idleness for the device.
Server Case
-----------
Overview
--------
Same as a desktop, except that interactivity be damned. You
want processes to _rarely_ have to wait for swap-ins, and
you want as much read-ahead as possible. Idle tasks are pressed
firmly into cache to make room for running processes.
Machine Description
-------------------
RAM: 512Mb-64Gb (reads: immediate, writes: immediate)
HD: 10Gb-4Tb (reads: medium, writes: medium)
Swap: 2*RAM size (HD speeds)
Motivations
-----------
* Keep running processes as fully in memory as possible.
----------------------------- SUMMARY ----------------------------------
If we take all the motivations from the above, and list them,
we get:
* Don't write to the (slow,packeted) devices until
you need to free up memory for processes.
* Never cache reads from immediate/fast devices.
* If we're getting low cache hit rates, don't flush
processes to swap.
* If we're getting good cache hit rates, flush old, idle
processes to swap.
* Keep packetized devices as continuously-idle as possible.
Small chunks of idleness don't count. You want to have
maximal stetches of idleness for the device.
* Keep running processes as fully in memory as possible.
Oddly enough, they don't seem to conflict. I'll continue to
work on these motivations, and try to determine testable methods
of measuring the success of a VM versus these criteria.
Comments welcome.
--
Jason McMullan, Senior Linux Consultant
Linuxcare, Inc. 412.432.6457 tel, 412.656.3519 cell
jmcmul...@linuxcare.com, http://www.linuxcare.com/
Linuxcare. Putting open source to work.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!sn-xit-02!supernews.com!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
X-Authentication-Warning: duckman.distro.conectiva: riel owned process doing -bs
Original-Date: Tue, 26 Jun 2001 18:21:21 -0300 (BRST)
From: Rik van Riel <r...@conectiva.com.br>
X-X-Sender: <r...@duckman.distro.conectiva>
To: Jason McMullan <jmcmul...@linuxcare.com>
Cc: <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
In-Reply-To: <20010626155838.A23098@jmcmullan.resilience.com>
Original-Message-ID: <Pine.LNX.4.33L.0106261819400.23373-100000@duckman.distro.conectiva>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Tue, 26 Jun 2001 21:22:39 GMT
Message-ID: <fa.or4qj9v.31msr5@ifi.uio.no>
References: <fa.l9ipguv.b249q3@ifi.uio.no>
Lines: 46
On Tue, 26 Jun 2001, Jason McMullan wrote:
> If we take all the motivations from the above, and list
> them, we get:
>
> * Don't write to the (slow,packeted) devices until
> you need to free up memory for processes.
> * Never cache reads from immediate/fast devices.
> * Keep packetized devices as continuously-idle as possible.
> Small chunks of idleness don't count. You want to have
> maximal stetches of idleness for the device.
> * Keep running processes as fully in memory as possible.
I agree with your modification, and with the obvious 4
points above ...
> * If we're getting low cache hit rates, don't flush
> processes to swap.
> * If we're getting good cache hit rates, flush old, idle
> processes to swap.
... but I fail to see this one. If we get a low cache hit
rate, couldn't that just mean we allocated too little memory
for the cache ?
I am very much interested in continuing this discussion...
Also, how would we translate all these requirements into
VM strategies ?
regards,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
X-Authentication-Warning: sekrit.casc.com: stoffel set sender to stof...@casc.com using -f
From: John Stoffel <stof...@casc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Original-Message-ID: <15160.65442.682067.38776@gargle.gargle.HOWL>
Original-Date: Tue, 26 Jun 2001 17:33:22 -0400
To: Rik van Riel <r...@conectiva.com.br>
Cc: Jason McMullan <jmcmul...@linuxcare.com>, <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
In-Reply-To: <Pine.LNX.4.33L.0106261819400.23373-100000@duckman.distro.conectiva>
Original-References: <20010626155838.A23...@jmcmullan.resilience.com>
<Pine.LNX.4.33L.0106261819400.23373-100...@duckman.distro.conectiva>
X-Mailer: VM 6.92 under Emacs 20.6.1
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Tue, 26 Jun 2001 21:34:48 GMT
Message-ID: <fa.hot0mfv.gksm1c@ifi.uio.no>
References: <fa.or4qj9v.31msr5@ifi.uio.no>
Lines: 34
>> * If we're getting low cache hit rates, don't flush
>> processes to swap.
>> * If we're getting good cache hit rates, flush old, idle
>> processes to swap.
Rik> ... but I fail to see this one. If we get a low cache hit rate,
Rik> couldn't that just mean we allocated too little memory for the
Rik> cache ?
Or that we're doing big sequential reads of file(s) which are larger
than memory, in which case expanding the cache size buys us nothing,
and can actually hurt us alot.
I personally don't feel that the cache should be allowed to grow over
50% of the system's memory at all, we've got so much in the cache at
that point, that we're probably not hitting it all that much.
This is why the discussion on the other cache scanning algorithm
(2Q+?) was so interesting, since it looked to handle both the LRU
vs. FIFO tradeoffs very nicely.
Rik> I am very much interested in continuing this discussion...
Me too, even if I can just contribute comments and not much code.
John
John Stoffel - Senior Unix Systems Administrator - Lucent Technologies
stof...@lucent.com - http://www.lucent.com - 978-952-7548
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
X-Authentication-Warning: duckman.distro.conectiva: riel owned process doing -bs
Original-Date: Tue, 26 Jun 2001 18:42:56 -0300 (BRST)
From: Rik van Riel <r...@conectiva.com.br>
X-X-Sender: <r...@duckman.distro.conectiva>
To: John Stoffel <stof...@casc.com>
Cc: Jason McMullan <jmcmul...@linuxcare.com>, <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
In-Reply-To: <15160.65442.682067.38776@gargle.gargle.HOWL>
Original-Message-ID: <Pine.LNX.4.33L.0106261838320.23373-100000@duckman.distro.conectiva>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Tue, 26 Jun 2001 21:44:19 GMT
Message-ID: <fa.oqkojpv.3hosb7@ifi.uio.no>
References: <fa.hot0mfv.gksm1c@ifi.uio.no>
Lines: 48
On Tue, 26 Jun 2001, John Stoffel wrote:
> >> * If we're getting low cache hit rates, don't flush
> >> processes to swap.
> >> * If we're getting good cache hit rates, flush old, idle
> >> processes to swap.
>
> Rik> ... but I fail to see this one. If we get a low cache hit rate,
> Rik> couldn't that just mean we allocated too little memory for the
> Rik> cache ?
>
> Or that we're doing big sequential reads of file(s) which are
> larger than memory, in which case expanding the cache size buys
> us nothing, and can actually hurt us alot.
That's a big "OR". I think we should have an algorithm to
see which of these two is the case, otherwise we're just
making the wrong decision half of the time.
Also, in many systems we'll be doing IO on _multiple_ files
at the same time, so I guess this will have to be a file-by-file
decision.
> I personally don't feel that the cache should be allowed to grow over
> 50% of the system's memory at all, we've got so much in the cache at
> that point, that we're probably not hitting it all that much.
Remember that disk cache includes stuff like mmap()ed
executables and swap-backed user memory. Do you really
want to limit those too ?
regards,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Message-ID: <3B399EF8.9BA76FA2@TeraPort.de>
Original-Date: Wed, 27 Jun 2001 10:53:12 +0200
From: Martin Knoblauch <Martin.Knobla...@TeraPort.de>
X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.4.5-ac18 i686)
X-Accept-Language: en, de
MIME-Version: 1.0
To: "linux-ker...@vger.kernel.org" <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: TeraPort GmbH
Date: Wed, 27 Jun 2001 08:56:10 GMT
Message-ID: <fa.f1oagev.b1k53e@ifi.uio.no>
Lines: 38
>> * If we're getting low cache hit rates, don't flush
>> processes to swap.
>> * If we're getting good cache hit rates, flush old, idle
>> processes to swap.
Rik> ... but I fail to see this one. If we get a low cache hit rate,
Rik> couldn't that just mean we allocated too little memory for the
Rik> cache ?
maybe more specific: If the hit-rate is low and the cache is already
70+% of the systems memory, the chances maybe slim that more cache is
going to improve the hit-rate.
I do not care much whether the cache is using 99% of the systems memory
or 50%. As long as there is free memory, using it for cache is great. I
care a lot if the cache takes down interactivity, because it pushes out
processes that it thinks idle, but that I need in 5 seconds. The caches
pressure against processes should decrease with the (relative) size of
the cache. Especially in low hit-rate situations.
OT: I asked the question before somewhere else. Are there interfaces to
the VM that expose the various cache sizes and, more important,
hit-rates to userland? I would love to see (or maybe help writing in my
free time) a tool to just visualize/analyze the efficiency of the VM
system.
Martin
--
------------------------------------------------------------------
Martin Knoblauch | email: Martin.Knobla...@TeraPort.de
TeraPort GmbH | Phone: +49-89-510857-309
C+ITS | Fax: +49-89-510857-111
http://www.teraport.de | Mobile: +49-170-4904759
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!news2.google.com!newsfeed.google.com!
newsfeed.stanford.edu!news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!
ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Message-ID: <3B3B14AB.DF020611@idb.hist.no>
Original-Date: Thu, 28 Jun 2001 13:27:39 +0200
From: Helge Hafting <helge.haft...@idb.hist.no>
X-Mailer: Mozilla 4.06 [en] (WinNT; I)
MIME-Version: 1.0
To: Martin Knoblauch <Martin.Knobla...@TeraPort.de>
CC: "linux-ker...@vger.kernel.org" <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
Original-References: <3B399EF8.9BA76...@TeraPort.de>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 11:30:56 GMT
Message-ID: <fa.eo4cndv.1luoj3@ifi.uio.no>
References: <fa.f1oagev.b1k53e@ifi.uio.no>
Lines: 34
Martin Knoblauch wrote:
>
> maybe more specific: If the hit-rate is low and the cache is already
> 70+% of the systems memory, the chances maybe slim that more cache is
> going to improve the hit-rate.
>
Oh, but this is posible. You can get into situations where
the (file cache) working set needs 80% or so of memory
to get a near-perfect hitrate, and where
using 70% of memory will trash madly due to the file access
pattern. And this won't be a problem either, if
the working set of "other" (non-file)
stuff is below 20% of memory. The total size of
non-file stuff may be above 20% though, so something goes
into swap.
I definitely want the machine to work under such circumstances,
so an arbitrary limit of 70% won't work.
Preventing swap-trashing at all cost doesn't help if the
machine loose to io-trashing instead. Performance will be
just as much down, although perhaps more satisfying because
people aren't that surprised if explicit file operations
take a long time. They hate it when moving the mouse
or something cause a disk access even if their
apps runs faster. :-(
Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Thu, 28 Jun 2001 14:02:09 +0200 (CEST)
From: Tobias Ringstrom <t...@unhappy.mine.nu>
X-X-Sender: <t...@boris.prodako.se>
To: Helge Hafting <helge.haft...@idb.hist.no>
cc: Martin Knoblauch <Martin.Knobla...@TeraPort.de>,
"linux-ker...@vger.kernel.org" <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
In-Reply-To: <3B3B14AB.DF020611@idb.hist.no>
Original-Message-ID: <Pine.LNX.4.33.0106281346280.1258-100000@boris.prodako.se>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 12:03:56 GMT
Message-ID: <fa.knel7sv.l7ab0g@ifi.uio.no>
References: <fa.eo4cndv.1luoj3@ifi.uio.no>
Lines: 34
On Thu, 28 Jun 2001, Helge Hafting wrote:
> Preventing swap-trashing at all cost doesn't help if the
> machine loose to io-trashing instead. Performance will be
> just as much down, although perhaps more satisfying because
> people aren't that surprised if explicit file operations
> take a long time. They hate it when moving the mouse
> or something cause a disk access even if their
> apps runs faster. :-(
Exactly. I still want the ability to tune the system according to my
taste. I've been thinking about this for some time, and I've specifically
tried to come up with nice tunables, completely ignoring if it is possible
now or not.
If individual pages could be classified as code (text segments), data,
file cache, and so on, I would specify costs to the paging of such pages
in or out. This way I can make the system perfer to drop a file cache
page that has not been accessed for five minutes, over a program text page
that has not been acccessed for one hour (or much more).
This would be very useful, I think. Would it be very hard to classify
pages like this (text/data/cache/...)?
Any reason why this is a bad idea?
/Tobias
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
To: <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
X-Mailer: Lotus Notes Release 5.0.5 September 22, 2000
From: mike_phill...@urscorp.com
Original-Message-ID: <OF95A43E53.39291B42-ON84256A79.003D421D@urscorp.com>
Original-Date: Thu, 28 Jun 2001 09:20:09 -0300
X-MIMETrack: Serialize by Router on SMTP102/URSCorp(Release 5.0.5 |September 22, 2000) at
06/28/2001 08:15:47 AM,
Serialize complete at 06/28/2001 08:15:47 AM
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 12:22:58 GMT
Message-ID: <fa.k8ijs9v.v6icr4@ifi.uio.no>
Lines: 28
> If individual pages could be classified as code (text segments),
> data, file cache, and so on, I would specify costs to the paging
> of such pages in or out. This way I can make the system perfer
> to drop a file cache page that has not been accessed for five
> minutes, over a program text page that has not been acccessed
> for one hour (or much more).
This would be extremely useful. My laptop has 256mb of ram, but every day
it runs the updatedb for locate. This fills the memory with the file
cache. Interactivity is then terrible, and swap is unnecessarily used. On
the laptop all this hard drive thrashing is bad news for battery life
(plus the fact that laptop hard drives are not the fastest around). I
purposely do not run more applications than can comfortably fit in the
256mb of memory.
If fact, to get interactivity back, I've got a small 10 liner that mallocs
memory to *force* stuff into swap purely so I can have a large block of
memory back for interactivity.
Something simple that did "you haven't used this file for 30mins, flush it
out of the cache would be sufficient"
Mike
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
To: mike_phill...@urscorp.com
Original-Date: Thu, 28 Jun 2001 13:30:44 +0100 (BST)
Cc: linux-ker...@vger.kernel.org
In-Reply-To: <OF95A43E53.39291B42-ON84256A79.003D421D@urscorp.com>
from "mike_phillips@urscorp.com" at Jun 28, 2001 09:20:09 AM
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Original-Message-Id: <E15FawW-0006qI-00@the-village.bc.nu>
From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 12:32:00 GMT
Message-ID: <fa.h53j0nv.1c7um9f@ifi.uio.no>
References: <fa.k8ijs9v.v6icr4@ifi.uio.no>
Lines: 15
> This would be extremely useful. My laptop has 256mb of ram, but every day
> it runs the updatedb for locate. This fills the memory with the file
> cache. Interactivity is then terrible, and swap is unnecessarily used. On
> the laptop all this hard drive thrashing is bad news for battery life
That isnt really down to labelling pages, what you are talking qbout is what
you get for free when page aging works right (eg 2.0.39) but don't get in
2.2 - and don't yet (although its coming) quite get right in 2.4.6pre.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
cyclone.bc.net!newsfeed.media.kyoto-u.ac.jp!uio.no!nntp.uio.no!ifi.uio.no!
internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Thu, 28 Jun 2001 15:33:57 +0200 (CEST)
From: Tobias Ringstrom <t...@unhappy.mine.nu>
X-X-Sender: <t...@boris.prodako.se>
To: Alan Cox <a...@lxorguk.ukuu.org.uk>
cc: <mike_phill...@urscorp.com>, <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
In-Reply-To: <E15FawW-0006qI-00@the-village.bc.nu>
Original-Message-ID: <Pine.LNX.4.33.0106281523390.1258-100000@boris.prodako.se>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 13:36:11 GMT
Message-ID: <fa.knur7cv.ln4agl@ifi.uio.no>
References: <fa.h53j0nv.1c7um9f@ifi.uio.no>
Lines: 28
On Thu, 28 Jun 2001, Alan Cox wrote:
> > This would be extremely useful. My laptop has 256mb of ram, but every day
> > it runs the updatedb for locate. This fills the memory with the file
> > cache. Interactivity is then terrible, and swap is unnecessarily used. On
> > the laptop all this hard drive thrashing is bad news for battery life
>
> That isnt really down to labelling pages, what you are talking qbout is what
> you get for free when page aging works right (eg 2.0.39) but don't get in
> 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre.
Correct, but all pages are not equal.
The problem with updatedb is that it pushes all applications to the swap,
and when you get back in the morning, everything has to be paged back from
swap just because the (stupid) OS is prepared for yet another updatedb
run.
Other bad activities include copying lots of files, tar/untar:ing and CD
writing. They all cause unwanted paging, at least for the desktop user.
/Tobias
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
feed.textport.net!newsfeed.media.kyoto-u.ac.jp!uio.no!nntp.uio.no!
ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
To: t...@unhappy.mine.nu (Tobias Ringstrom)
Original-Date: Thu, 28 Jun 2001 14:37:20 +0100 (BST)
Cc: a...@lxorguk.ukuu.org.uk (Alan Cox), mike_phill...@urscorp.com,
linux-ker...@vger.kernel.org
In-Reply-To: <Pine.LNX.4.33.0106281523390.1258-100000@boris.prodako.se>
from "Tobias Ringstrom" at Jun 28, 2001 03:33:57 PM
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Original-Message-Id: <E15Fbyy-0006xF-00@the-village.bc.nu>
From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 13:38:43 GMT
Message-ID: <fa.h93h97v.197stpf@ifi.uio.no>
References: <fa.knur7cv.ln4agl@ifi.uio.no>
Lines: 28
> > That isnt really down to labelling pages, what you are talking qbout is what
> > you get for free when page aging works right (eg 2.0.39) but don't get in
> > 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre.
>
> Correct, but all pages are not equal.
That is the whole point of page aging done right. The use of a page dictates
how it is aged before being discarded. So pages referenced once are aged
rapidly, but once they get touched a couple of times then you know they arent
streaming I/O. There are other related techniques like punishing pages that
are touched when streaming I/O is done to pages further down the same file -
FreeBSD does this one for example
> The problem with updatedb is that it pushes all applications to the swap,
> and when you get back in the morning, everything has to be paged back from
> swap just because the (stupid) OS is prepared for yet another updatedb
> run.
Updatedb is a bit odd in that it mostly sucks in metadata and the buffer to
page cache balancing is a bit suspect IMHO.
Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Thu, 28 Jun 2001 16:04:34 +0200 (CEST)
From: Tobias Ringstrom <t...@unhappy.mine.nu>
X-X-Sender: <t...@boris.prodako.se>
To: Alan Cox <a...@lxorguk.ukuu.org.uk>
cc: <mike_phill...@urscorp.com>, <linux-ker...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
In-Reply-To: <E15Fbyy-0006xF-00@the-village.bc.nu>
Original-Message-ID: <Pine.LNX.4.33.0106281558250.1258-100000@boris.prodako.se>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 14:05:57 GMT
Message-ID: <fa.knej84v.l7sb8u@ifi.uio.no>
References: <fa.h93h97v.197stpf@ifi.uio.no>
Lines: 37
On Thu, 28 Jun 2001, Alan Cox wrote:
> > > That isnt really down to labelling pages, what you are talking qbout is what
> > > you get for free when page aging works right (eg 2.0.39) but don't get in
> > > 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre.
> >
> > Correct, but all pages are not equal.
>
> That is the whole point of page aging done right. The use of a page dictates
> how it is aged before being discarded. So pages referenced once are aged
> rapidly, but once they get touched a couple of times then you know they arent
> streaming I/O. There are other related techniques like punishing pages that
> are touched when streaming I/O is done to pages further down the same file -
> FreeBSD does this one for example
Are you saying that classification of pages will not be useful?
Only looking at the page access patterns can certainly reveal a lot, but
tuning how to punish different pages is useful.
> > The problem with updatedb is that it pushes all applications to the swap,
> > and when you get back in the morning, everything has to be paged back from
> > swap just because the (stupid) OS is prepared for yet another updatedb
> > run.
>
> Updatedb is a bit odd in that it mostly sucks in metadata and the buffer to
> page cache balancing is a bit suspect IMHO.
In 2.4.6-pre, the buffer cache is no longer used for metata, right?
/Tobias
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!
news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
To: t...@unhappy.mine.nu (Tobias Ringstrom)
Original-Date: Thu, 28 Jun 2001 15:14:19 +0100 (BST)
Cc: a...@lxorguk.ukuu.org.uk (Alan Cox), mike_phill...@urscorp.com,
linux-ker...@vger.kernel.org
In-Reply-To: <Pine.LNX.4.33.0106281558250.1258-100000@boris.prodako.se>
from "Tobias Ringstrom" at Jun 28, 2001 04:04:34 PM
X-Mailer: ELM [version 2.5 PL3]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Original-Message-Id: <E15FcYl-00070C-00@the-village.bc.nu>
From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 28 Jun 2001 14:15:58 GMT
Message-ID: <fa.g5hb5vv.dlmohe@ifi.uio.no>
References: <fa.knej84v.l7sb8u@ifi.uio.no>
Lines: 11
> > Updatedb is a bit odd in that it mostly sucks in metadata and the buffer to
> > page cache balancing is a bit suspect IMHO.
>
> In 2.4.6-pre, the buffer cache is no longer used for metata, right?
For ext2 directory blocks the page cache is now used
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/