Lab 3: Dynamic Memory Allocation

Assigned: Oct 9, Due: Oct 26, 11:59 PM

The TAs for this lab are: Hang Yan, Kai Shen

Introduction

OptsRus is doing really well now and has been asked to create a dynamic memory allocator (e.g., malloc, free and realloc routines) for C programs for a new experimental operating system. You are asked to explore the design of allocators and then implement an allocator that is correct, efficient and fast. This is a difficult lab and you must plan your time wisely.

Setup

Start by copying the ece454-lab3.tar.gz file from the shared directory /cad2/ece454f/hw3/ on the UG machines in a protected directory in your UG home directory. Then run the command:

tar xzvf ece454-lab3.tar.gz

The mdriver program is a driver program that allows you to evaluate the performance of your solution. Use the make command to generate the driver code and run it with the command ./mdriver -V. The -V flag displays helpful summary information.

The only file you will be modifying and submitting is mm.c. In this file, please add the requested information in the team C structure so that the automarker can identify you. Do this right away so you don't forget.

Allocator API

Your dynamic memory allocator will consist of the following four functions, which are declared in mm.h and defined in mm.c.

int mm_init(void);
void *mm_malloc(size_t size);
void mm_free(void *ptr);
void *mm_realloc(void *ptr, size_t size);

The mm.c file we have provided implements the simplest versions of these functions that are functionally correct. You need to modify these functions (and possibly define other private static functions), so that they obey the following semantics:

• mm_init: Before calling mm_malloc, mm_realloc or mm_free, the application program (i.e., the trace-driven driver program that you will use to evaluate your implementation) calls mm_init to perform any necessary initialization, such as allocating the initial heap area. The return value of the function should be 0, unless there is a problem performing the initialization, in which case, it should be -1.

• mm_malloc: The mm_malloc routine returns a pointer to an allocated block payload of at least size bytes. The entire allocated block should lie within the heap region and should not overlap with any other allocated chunk. Since the malloc implementation in the standard C library (libc) always returns payload pointers that are aligned to 16 bytes on the x86_64 architecture, your implementation should do likewise and return 16-byte aligned pointers.

• mm_free: The mm_free routine frees the block pointed to by ptr. It returns nothing. This routine is only guaranteed to work when the passed pointer (ptr) was returned by an earlier call to mm_malloc or mm_realloc and has not yet been freed.

• mm_realloc: The mm_realloc routine returns a pointer to an allocated region of at least size bytes with the following constraints.

  • if ptr is NULL, the call is equivalent to mm_malloc(size);
  • if size is equal to zero, the call is equivalent to mm_free(ptr);
  • if ptr is not NULL, it must have been returned by an earlier call to mm_malloc or mm_realloc. The call to mm_realloc changes the size of the memory block pointed to by ptr (the old block) to size bytes and returns the address of the new block.
  • The address of the new block might be the same as the old block, or it might be different, depending on your implementation, the amount of internal fragmentation in the old block, and the size of the reallocation request.
  • The contents of the new block are the same as those of the old block, up to the minimum of the old and new sizes. Everything else is uninitialized. For example, if the old block is 16 bytes and the new block is 24 bytes, then the first 16 bytes of the new block are identical to the first 16 bytes of the old block and the last 8 bytes are uninitialized. Similarly, if the old block is 24 bytes and the new block is 16 bytes, then the contents of the new block are identical to the first 16 bytes of the old block.

These semantics match the semantics of the corresponding malloc, realloc, and free routines in libc. Type man malloc in the shell for complete documentation.

Heap Consistency Checker

Dynamic memory allocators are notoriously tricky to program correctly and efficiently. They are difficult to program correctly because they involve a lot of untyped pointer manipulation. You will find it helpful to write a heap checker that scans the heap and checks it for consistency. Some examples of what a heap checker might check are:

• Is every block in the free list marked as free?
• Are there any contiguous free blocks that escaped coalescing?
• Is every free block actually in the free list?
• Do the pointers in the free list point to valid free blocks?
• Do any allocated blocks overlap?
• Do the pointers in a heap block point to valid heap addresses?

Your heap checker will consist of the function int mm_check(void) in mm.c. It should check any invariants or consistency conditions you consider important. It returns a non-zero value if and only if your heap is consistent. You are not limited to the listed suggestions and you are not required to check all of them. You are encouraged to print out error messages when mm_check fails.

This consistency checker is for your own debugging during development. When you submitmm.c, make sure to comment out any calls to mm_check as they will lower throughput.

Support Routines

The memlib.c file simulates the memory system for your dynamic memory allocator. You can invoke the following functions declared in memlib.h:

• void *mem_sbrk(int incr): Expands the heap by incr bytes, where incr is a positive non-zero integer and returns a generic pointer to the first byte of the newly allocated heap area. The semantics of this function are identical to the Unix sbrk system call, except that mem_sbrk only accepts a positive non-zero integer argument.

• void *mem_heap_lo(void): Returns a generic pointer to the first byte in the heap.

• void *mem_heap_hi(void): Returns a generic pointer to the last byte in the heap.
• size_t mem_heapsize(void): Returns the current size of the heap in bytes.
• size_t mem_pagesize(void): Returns the system's page size in bytes (4K on Linux systems).

The Driver Program

The mdriver driver program tests your mm.c implementation for correctness, memory utilization, and throughput. The driver program takes one or more trace files as input. A trace file contains a sequence of allocate, reallocate, and free requests that instruct the driver to call your mm_malloc, mm_realloc, and mm_free routines. The mdriver program accepts the following command line arguments:

• -t <tracedir>: Look for trace files in directory tracedir instead of in the default ../traces directory.
• -f <tracefile>: Use this tracefile for testing instead of the default set of tracefiles.
• -h: Print a summary of the command line arguments.
• -l: Run and measure the malloc implementation in libc in addition to the student's implementation.
• -v: Verbose output. Print performance breakdown for each tracefile in a compact table.
• -V: More verbose output. Prints additional diagnostic information as each trace file is processed. Useful during debugging for determining which trace file is causing your implementation to fail.

Dynamic Allocator

Your main task in this lab is to design a dynamic allocator that implements the Allocator API. Here are the rules for this lab:

• You should not change any of the allocator interfaces declared in mm.h.
• You are not allowed to invoke any memory-management related library calls or system calls in your code, e.g., malloc, calloc, free, realloc, sbrk, brk or any variants of these calls. However, you are allowed to use memcpy and memmove.
• You are not allowed to use mmap or related calls to create new virtual address regions.
• The total size of all defined global and static scalar variables and compound data structures should be small. This memory should only be used for the minimal amount of metadata needed to bootstrap your memory allocator rather than for allocations.
• For consistency with the malloc implementation in libc on the x86_64 architecture, your allocator must always return pointers that are aligned to 16-byte boundaries. The driver enforces this requirement.

Evaluation

The total grade for this assignment is 100 points. You will receive zero points if you break any of the rules described above. Otherwise, your grade will be calculated based on two performance metrics:

• Memory Utilization (U): The ratio of the maximum allocated memory (i.e., memory allocated via mm_malloc or mm_realloc but not yet freed via mm_free) to the maximum heap size used by your implementation. We calculate the allocated memory and the heap size after each request. Then we calculate the maximum value of the allocated memory and the heap size over the entire run. The optimal memory utilization is 1. Both internal and external fragmentation and header space will reduce memory utilization. You should aim for memory utilization to be as close to optimal as possible.

• Throughput (T): The average number of operations completed per second.

The mdriver program outputs the performance of your allocator in terms of a weighted sum of memory utilization and throughput. It computes a performance index that lies between [0, 100] as follows:

P=⌊wU⌋ + ⌊(100-w) * min(1, T/Tc)⌋

where w is a weight, U is memory utilization, T is throughput, and Tc is the estimated throughput of the libc malloc implementation on your system for the default traces. We set w=60 to slightly favor memory utilization over throughput.

The driver computes the performance index P for each trace that passes validation. This value is then scaled down (linearly) by the fraction of traces that pass validation. For example, if half of the traces pass validation, P will be divided by two.

If your code is buggy and crashes the driver when running all traces together (i.e., without the -f option), you will get a zero mark. So make sure that your code does not break the driver.

FAQ

For more details about the lab, read the Lab 3 FAQ.

Submission

Submit your assignment by typing submitece454f 3 mm.con one of the UG EECG machines. Do not submit any other files since we do not use them.

We will use an automarker for this lab to mark your lab (similar to Lab 2). The automarker URL will be provided on Piazza within a week of releasing this lab. The automarker may give a mark close to 100% but not 100%. Therefore, after the lab is done, we will increase everyone's mark by a constant score so that the top student gets 100% in this lab.