VM: fix pageout throttling to external pagers

Since the VM system has been tracking whether pages belong to internal
or external objects, pageout throttling to external pagers has simply
not been working. The reason is that, on pageout, requests for external
pages are correctly tracked, but on page release (which is used to
acknowledge the request), external pages are not marked external
any more. This is because the external bit tracks whether a page
belongs to an external object, and all pages, including external
ones, are moved to an internal object during pageout.

To solve this issue, a new "external_laundry" bit is added. It has
the same purpose as the laundry bit, but for external pagers.

* vm/vm_page.c (vm_page_seg_min_page_available): Function unused, remove.
(vm_page_seg_evict): Use vm_page_external_laundry_count instead of
vm_page_external_pagedout. Add an assertion about double paging.
(vm_page_check_usable): Use vm_page_external_laundry_count instead of
vm_page_external_pagedout.
(vm_page_evict): Likewise.
* vm/vm_page.h (struct vm_page): New `external_laundry' member.
(vm_page_external_pagedout): Rename to ...
(vm_page_external_laundry_count): ... this.
* vm/vm_pageout.c: Include kern/printf.h.
(DEBUG): New macro.
(VM_PAGEOUT_TIMEOUT): Likewise.
(vm_pageout_setup): Use vm_page_external_laundry_count instead of
vm_page_external_pagedout. Set `external_laundry' where appropriate.
(vm_pageout): Use VM_PAGEOUT_TIMEOUT with thread_set_timeout.
Add debugging code, commented out by default.
* vm/vm_resident.c (vm_page_external_pagedout): Rename to ...
(vm_page_external_laundry_count): ... this.
(vm_page_init_template): Set `external_laundry' member to FALSE.
(vm_page_release): Rename external parameter to external_laundry.
Slightly change pageout resuming.
(vm_page_free): Rename external variable to external_laundry.

1 files changed, 5 insertions, 9 deletions

diff --git a/vm/vm_page.c b/vm/vm_page.c
index 94439b51..92e36a1a 100644
--- a/vm/vm_page.c
+++ b/vm/vm_page.c
@@ -925,12 +925,6 @@ vm_page_seg_pull_cache_page(struct vm_page_seg *seg,
 }
 
 static boolean_t
-vm_page_seg_min_page_available(const struct vm_page_seg *seg)
-{
-    return (seg->nr_free_pages > seg->min_free_pages);
-}
-
-static boolean_t
 vm_page_seg_page_available(const struct vm_page_seg *seg)
 {
     return (seg->nr_free_pages > seg->high_free_pages);
@@ -1099,6 +1093,7 @@ vm_page_seg_evict(struct vm_page_seg *seg, boolean_t external_only,
 
     page = NULL;
     object = NULL;
+    laundry = FALSE;
 
 restart:
     vm_page_lock_queues();
@@ -1156,6 +1151,7 @@ restart:
      */
 
     assert(!page->laundry);
+    assert(!(laundry && page->external));
 
     if (object->internal || !alloc_paused) {
         laundry = FALSE;
@@ -1878,7 +1874,7 @@ vm_page_check_usable(void)
         }
     }
 
-    vm_page_external_pagedout = -1;
+    vm_page_external_laundry_count = -1;
     vm_page_alloc_paused = FALSE;
     thread_wakeup(&vm_page_alloc_paused);
     return TRUE;
@@ -1980,7 +1976,7 @@ vm_page_evict(boolean_t *should_wait)
     external_only = TRUE;
 
     simple_lock(&vm_page_queue_free_lock);
-    vm_page_external_pagedout = 0;
+    vm_page_external_laundry_count = 0;
     alloc_paused = vm_page_alloc_paused;
     simple_unlock(&vm_page_queue_free_lock);
 
@@ -2008,7 +2004,7 @@ again:
      * Keep in mind eviction may not cause pageouts, since non-precious
      * clean pages are simply released.
      */
-    if ((vm_page_external_pagedout == 0) && (vm_page_laundry_count == 0)) {
+    if ((vm_page_laundry_count == 0) && (vm_page_external_laundry_count == 0)) {
         /*
          * No pageout, but some clean pages were freed. Start a complete
          * scan again without waiting.