Difference between revisions of "Memory"

From Embedded Xinu
Jump to navigation Jump to search
(Added <tt> tags to addresses)
m (→‎Kernel Segments: More emphasis on mapping and caching properties)
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
Memory on the [[WRT54GL]] is still (on the whole) not fully understood, this page is dedicated to rectifying that problemView the [[Talk:Memory|talk]] page for a simple (possibly outdated) dump of what memory should look like.
+
Memory on MIPS-based processors is broken into several segments, consuming the entire 32-address spaceThese segments are arranged as follows:
  
== Introduction ==
+
* [[#User Segment|User Segment]] (USEG), 2 GB '''mapped''' and '''cached''', addresses <tt>0x0000 0000</tt> through <tt>0x7FFF FFFF</tt>
Memory on almost every MIPS processor is broken into two major sections---user memory and kernel memory.  Each of these sections occupies two gigabytes of memory and provides a simple mapping from code addresses to physical addresses which will be explained later.
+
* [[#Kernel Segment 0|Kernel Segment 0]] (KSEG0), 512 MB '''unmapped''' and '''cached''', addresses <tt>0x8000 0000</tt> through <tt>0x9FFF FFFF</tt>
 +
* [[#Kernel Segment 1|Kernel Segment 1]] (KSEG1), 512 MB '''unmapped''' and '''uncached''', addresses <tt>0xA000 0000</tt> through <tt>0xBFFF FFFF</tt>
 +
* [[#Kernel Segment 2|Kernel Segment 2]] (KSEG2), 1 GB '''mapped''' and '''cached''', addresses <tt>0xC000 0000</tt> through <tt>0xFFFF FFFF</tt>
  
=== User Memory ===
+
Note that the [[WRT54GL]] only has 16 MB of main memory, so a 1-1 mapping is not be available above <tt>0x..FF FFFF</tt>.
User memory, also known as KUSEG, ranges from <tt>0x0000 0000</tt> to <tt>0x7FFF FFFF</tt> and can only be accessed when the processor is running in user-mode or (more dangerously) if the error level bit in the status register of co-processor 0 is set.  While executing in kernel-mode or supervisor-mode memory references to this segment will result in TLB exceptions.
 
  
Since the WRT54GL routers only have 16 megabytes of RAM, it is currently assumed that <tt>0x0000 0000</tt>--<tt>0x00FF FFFF</tt> has a 1-1 mapping with physical memory.
+
== User Segment ==
 +
 
 +
The user segment of memory, or USEG, is the range of memory addresses from <tt>0x0000 0000</tt> through <tt>0x7FFF FFFF</tt>.  This memory is both mapped and cached, meaning that any attempt to access memory within this range will result in the hardware consulting the memory manager prior to access.  '''Note''' that any attempted access in this range must with the CPU privilege level set to user mode (i.e. <math>Status(KSU) = 2</math>) or the processor must be in the exception state with the error level bit set (i.e. <math>Status(EXL) = 1</math>).  In the latter case, all mappings turn into 1-1, so directly accessing these address is not recommended.
 +
 
 +
Under normal operation, when an address in this memory range is accessed, the processor will query the memory management unit is consulted for a mapping.  In turn this will ask the translation lookaside buffer (TLB), and if no mapping is found cause a TLB load or store exception.  When this exception occurs, the operating system is consulted and should write the correct mapping to the TLB and return from the exception handler.
 +
 
 +
It should be noted that when a TLB exception occurs in the USEG range of addresses, a special "fast" exception handler is consulted instead of the normal exception handler.  This fast handler is at most 32 instructions long and begins at <tt>0x8000 0000</tt>.
 +
 
 +
=== User Segment under Xinu ===
 +
 
 +
[[Embedded Xinu]] has basic [[memory management]] as of the 2.0 release (stored in the <tt>mem/</tt> directory).  During initialization, the kernel allocates some amount of memory (defined in <code>xinu.conf</code> as UHEAP_SIZE) to act as the user memory heap.  Once this memory is initialized, calls to <code>malloc</code> and <code>free</code> will use the user heap for memory allocation and automatically insert mappings into the system page table.  All mappings are 1-1 since there is no backing store for a virtual memory subsystem, though it would be possible to provide each thread with a private address space this requires lots of memory overhead.
 +
 
 +
Exception code to handle TLB faults is in the <code>tlbMissHandler</code> function and simply performs a lookup of the faulting address in the system page table, checks to see if it is valid and in the correct address space, and inserts the mapping in the TLB hardware.
 +
 
 +
== Kernel Segments ==
 +
 
 +
While the first part of memory is dedicated to the user segment, the remainder is the kernel segment.  Unlike the user segment, the kernel segment is sub-divided into three segments with different memory access properties.  '''Note''' that these properties can be very important if you are dealing with device drivers and/or hardware that uses direct memory access (DMA) because there may be caching considerations.
 +
 
 +
In brief:
  
=== Kernel Memory ===
+
* [[#Kernel Segment 0|KSEG0]] uses '''unmapped''' and '''cached''' memory accesses,
Kernel memory occupies the processor memory range from <tt>0x8000 0000</tt> to <tt>0xFFFF FFFF</tt> and is sub-divided into 3 smaller regions.  These regions are:
+
* [[#Kernel Segment 1|KSEG1]] uses '''unmapped''' and '''uncached''' memory accesses, and
* KSEG0 -- unmapped, cached memory (512 megabytes, starting at <tt>0x8000 0000</tt>),
+
* [[#Kernel Segment 2|KSEG2]] uses '''mapped''' and '''cached''' memory accesses.
* KSEG1 -- unmapped, uncached memory (512 megabytes, starting at <tt>0xA000 0000</tt>),
 
* KSEG2 -- mapped memory (1 gigabytes, starting at <tt>0xC000 0000</tt>).
 
  
It is important to notice that all direct memory accesses (DMAs) occur in the unmapped, uncached region of memory since devices often produce volatile data which would be dangerous to cache.
+
=== Kernel Segment 0 ===
  
Since physical memory is shared between user and kernel processes the mapping for KUSEG is simply 1-1, KSEG0 is <tt>address - 0x8000 0000</tt>, KSEG1 is <tt>address - 0xA000 0000</tt>, and finally KSEG2 is <tt>address - 0xC000 0000</tt>.
+
The first kernel segment, or KSEG0, is the range of memory addresses from <tt>0x8000 0000</tt> through <tt>0x9FFF FFFF</tt>.  This memory '''unmapped''' and '''cached''', meaning that when the processor attempts to access an address in this range it will not consult the memory manager for a mapping, but will store and modifications in the on-chip memory cache.  '''Note''' that when allocating memory for a device driver that will be using DMA, the caching effects could lead to major headaches.  If memory is being given to a hardware backend it should be mapped into the range of [[#Kernel Segment 1|KSEG1]].
  
== User Segment ==
+
On many MIPS processors the first page (4096 bytes) of KSEG0 is considered to be reserved system space where small amounts of specialized code can be loaded for fast execution.  Typically, this memory is used for exception handling and the following sections are reserved for the following:
As discussed above the user segment of memory occupies addresses from <tt>0x0000 0000</tt> to <tt>0x7FFF FFFF</tt>. Since physical memory is shared between each of the segments it should be noted that the first few pages of the user segment will actually be kernel code and should not be writable to user-level processesAlso the NULL pointer (<tt>0x0000 0000</tt>) could be considered a valid pointer in user-mode and this should be avoided so NULL pointer references are not ignored.
+
 
 +
* <tt>0x8000 0000</tt> (32 instructions) is for the TLB exception handler,
 +
* <tt>0x8000 0080</tt> (32 instructions) is for the 64-bit TLB exception handler, and
 +
* <tt>0x8000 0180</tt> (32 instructions) is for the generic exception handler.
 +
 
 +
Other reserved portions of this memory page remain unknown.
 +
 
 +
After the reserved system page, the operating system is free to use memory however it sees fit.
 +
 
 +
==== KSEG0 under Xinu ====
 +
 
 +
[[Embedded Xinu]] uses KSEG0 extensively for kernel operations.  As the WRT54GL uses a 32-bit MIPS processor, Embedded Xinu loads a quick TLB handler into memory at <tt>0x8000 0000</tt> and a generic exception handler at <tt>0x8000 0180</tt>, both of these are limited to 32 instructions and jump to higher level C code when needed. It is important to note that Embedded Xinu also makes use of reserved memory starting at <tt>0x8000 0200</tt> to store an array of exception handler entry points (32-bit function pointers for 32 possible exceptions) and <tt>0x8000 0280</tt> to store an array of interrupt handler entry points (32-bit function pointers for 8 possible interrupts).
 +
 
 +
Xinu loads the kernel entry point beginning at <tt>0x8000 1000</tt>, and upon booting begins execution at that address.  Within Embedded Xinu, the memory segments are in the following order (as defined by <code>ld.script</code>): text, read-only data, data, and block started by symbol (BSS, uninitialized data)The kernel allocates a small kernel stack after the BSS segment and finally initializes a dynamic memory heap for the remaining physical memory addresses.
 +
 
 +
The kernel memory allocator (<code>memget</code> and <code>memfree</code>), will allocate memory from the kernel heap as requested.  Since Embedded Xinu uses a single page table, all kernel addresses will be mapped read-only in all address spaces, giving a user thread the ability to read from but not write to kernel memory.
 +
 
 +
=== Kernel Segment 1 ===
 +
 
 +
The second kernel segment, or KSEG1, is the range of memory addresses from 0xA000 0000 through 0xBFFF FFFF. This memory '''unmapped''' and '''uncached''', meaning that when the processor attempts to access an address in this range it will not consult the memory manager for a mapping and it ''will'' bypass the on-chip memory cache for memory loads and stores.
 +
 
 +
By skipping the hardware cache, KSEG1 will see slower memory accesses because it must get data directly from the RAM.  Because of this, it is not typical to use KSEG1 for normal kernel operations, rather this segment is useful for accessing memory that is mapped to some other hardware device on the platform.  These mappings will either be pre-existing, so they are out-of-range of physical memory addresses, or they will be dynamically allocated memory that will be shared between the operating system and some hardware device.
 +
 
 +
==== KSEG1 under Xinu ====
 +
 
 +
Embedded Xinu uses several hardware devices that are mapped out-of-range of physical memory and some hardware devices that use dynamically allocated memory for sharing. Some devices on the WRT54GL that are beyond the range of physical memory are:
  
[[Embedded XINU]] does not currently have support for user-mode operations, so not much is known about the user segment.
+
* Broadcom I/O controllor registers at <tt>0xB800 0000</tt>,
 +
* [[UART]] registers at <tt>0xB800 0300</tt> and <tt>0xB800 0400</tt>,
 +
* Broadcom Ethernet 47xx registers at <tt>0xB800 1000</tt>,
 +
* Broadcom Wireless LAN controller registers at <tt>0xB800 5000</tt>,
 +
* Broadcom 47xx RoboSwitch registers at <tt>0xB800 6000</tt>, and
 +
* [[Flash memory]] (4 MB) read mapped beginning at <tt>0xBC00 0000</tt>.
  
== Kernel Segment 0 ==
+
Certain drivers (such as the [[Ethernet]] driver), also take advantage of shared memory between the operating system and the hardware.  This requires the use of dynamically allocated kernel memory (originating in KSEG0), that has been mapped to KSEG1 address range.  This is not problematic because both KSEG0 and KSEG1 use a 1-1 memory mapping.  With the Ethernet driver of Embedded Xinu, the shared memory that is in KSEG1 hold the DMA descriptor rings and the Ethernet packet buffers to store the packets in.
This segment exists from memory address <tt>0x8000 0000 to <tt>0x9FFF FFFF</tt> and is cached but remains unmapped (beyond the simple address - <tt>0x8000 0000</tt> mapping).   
 
  
The first 0x1000 bytes of this segment are reserved system space and contains small amounts of code for the [[interrupt handler]].  When an interrupt or exception occurs, the MIPS processor will jump to code located at <tt>0x8000 0180</tt>, expecting to find handler code at that location.  For safety reasons that code can only consume 0x20 bytes of memory (enough to jump to a safer, more robust handler).  [[Embedded XINU]] takes advantage of the space immediately after the end of interrupt handling code by loading entry vectors for the various interrupts and exceptions for speedy lookup.
+
=== Kernel Segment 2 ===
  
After the reserved system space it is safe to load generic code, XINU code is loaded to <tt>0x8000 1000</tt> and begins execution at the same offset. XINU code begins with the text segment, followed by the data and BSS segments.  After the compiled image is loaded into RAM, XINU allocates a specific amount of memory for the kernel stack immediately after the BSS segment.  Once the kernel stack has been setup, XINU initializes the heap beginning directly above the kernel stack and continuing until the end of physical memory (<tt>0x80FF FFFF</tt>).
+
The third kernel segment, or KSEG2, is the range of memory addresses from 0xC000 0000 through 0xFFFF FFFF. This memory is both '''mapped''' and '''cached''', meaning that the processor will consult the memory manager for a mapping and store memory modifications in the on-chip cache.
  
Kernel memory allocation will take memory addresses from the heap initialized in this segment.
+
Like the user segment of memory any attempt to access memory in KSEG2 will result in the processor querying the memory manager and the TLB to find a mapping.  If a mapping does not exist the processor will generate a TLB load or store exception and the operating system must fill the TLB entry.  Unlike USEG, a TLB exception will not jump to the "fast" handler and instead follow the normal path for exception handling through the generic exception mechanism.
  
== Kernel Segment 1 ==
+
This memory segment could be useful to create the appearance of page aligned data to the underlying hardware or operating system if needed.
  
=== Flash Memory ===
+
==== KSEG2 under Xinu ====
  
== Kernel Segment 2 ==
+
Embedded Xinu does not make use of any KSEG2 memory yet.  However, to take advantage of the Context register of MIPS processors when a TLB exception occurs, it is possible that a mapping of the system page table to KSEG2 might exist in future versions.
  
 
== References ==
 
== References ==
 
Sweetman, Dominic. ''See MIPS Run''. San Francisco: Morgan Kaufmann Publishers, 2007.
 
Sweetman, Dominic. ''See MIPS Run''. San Francisco: Morgan Kaufmann Publishers, 2007.

Latest revision as of 04:23, 29 July 2009

Memory on MIPS-based processors is broken into several segments, consuming the entire 32-address space. These segments are arranged as follows:

  • User Segment (USEG), 2 GB mapped and cached, addresses 0x0000 0000 through 0x7FFF FFFF
  • Kernel Segment 0 (KSEG0), 512 MB unmapped and cached, addresses 0x8000 0000 through 0x9FFF FFFF
  • Kernel Segment 1 (KSEG1), 512 MB unmapped and uncached, addresses 0xA000 0000 through 0xBFFF FFFF
  • Kernel Segment 2 (KSEG2), 1 GB mapped and cached, addresses 0xC000 0000 through 0xFFFF FFFF

Note that the WRT54GL only has 16 MB of main memory, so a 1-1 mapping is not be available above 0x..FF FFFF.

User Segment

The user segment of memory, or USEG, is the range of memory addresses from 0x0000 0000 through 0x7FFF FFFF. This memory is both mapped and cached, meaning that any attempt to access memory within this range will result in the hardware consulting the memory manager prior to access. Note that any attempted access in this range must with the CPU privilege level set to user mode (i.e. <math>Status(KSU) = 2</math>) or the processor must be in the exception state with the error level bit set (i.e. <math>Status(EXL) = 1</math>). In the latter case, all mappings turn into 1-1, so directly accessing these address is not recommended.

Under normal operation, when an address in this memory range is accessed, the processor will query the memory management unit is consulted for a mapping. In turn this will ask the translation lookaside buffer (TLB), and if no mapping is found cause a TLB load or store exception. When this exception occurs, the operating system is consulted and should write the correct mapping to the TLB and return from the exception handler.

It should be noted that when a TLB exception occurs in the USEG range of addresses, a special "fast" exception handler is consulted instead of the normal exception handler. This fast handler is at most 32 instructions long and begins at 0x8000 0000.

User Segment under Xinu

Embedded Xinu has basic memory management as of the 2.0 release (stored in the mem/ directory). During initialization, the kernel allocates some amount of memory (defined in xinu.conf as UHEAP_SIZE) to act as the user memory heap. Once this memory is initialized, calls to malloc and free will use the user heap for memory allocation and automatically insert mappings into the system page table. All mappings are 1-1 since there is no backing store for a virtual memory subsystem, though it would be possible to provide each thread with a private address space this requires lots of memory overhead.

Exception code to handle TLB faults is in the tlbMissHandler function and simply performs a lookup of the faulting address in the system page table, checks to see if it is valid and in the correct address space, and inserts the mapping in the TLB hardware.

Kernel Segments

While the first part of memory is dedicated to the user segment, the remainder is the kernel segment. Unlike the user segment, the kernel segment is sub-divided into three segments with different memory access properties. Note that these properties can be very important if you are dealing with device drivers and/or hardware that uses direct memory access (DMA) because there may be caching considerations.

In brief:

  • KSEG0 uses unmapped and cached memory accesses,
  • KSEG1 uses unmapped and uncached memory accesses, and
  • KSEG2 uses mapped and cached memory accesses.

Kernel Segment 0

The first kernel segment, or KSEG0, is the range of memory addresses from 0x8000 0000 through 0x9FFF FFFF. This memory unmapped and cached, meaning that when the processor attempts to access an address in this range it will not consult the memory manager for a mapping, but will store and modifications in the on-chip memory cache. Note that when allocating memory for a device driver that will be using DMA, the caching effects could lead to major headaches. If memory is being given to a hardware backend it should be mapped into the range of KSEG1.

On many MIPS processors the first page (4096 bytes) of KSEG0 is considered to be reserved system space where small amounts of specialized code can be loaded for fast execution. Typically, this memory is used for exception handling and the following sections are reserved for the following:

  • 0x8000 0000 (32 instructions) is for the TLB exception handler,
  • 0x8000 0080 (32 instructions) is for the 64-bit TLB exception handler, and
  • 0x8000 0180 (32 instructions) is for the generic exception handler.

Other reserved portions of this memory page remain unknown.

After the reserved system page, the operating system is free to use memory however it sees fit.

KSEG0 under Xinu

Embedded Xinu uses KSEG0 extensively for kernel operations. As the WRT54GL uses a 32-bit MIPS processor, Embedded Xinu loads a quick TLB handler into memory at 0x8000 0000 and a generic exception handler at 0x8000 0180, both of these are limited to 32 instructions and jump to higher level C code when needed. It is important to note that Embedded Xinu also makes use of reserved memory starting at 0x8000 0200 to store an array of exception handler entry points (32-bit function pointers for 32 possible exceptions) and 0x8000 0280 to store an array of interrupt handler entry points (32-bit function pointers for 8 possible interrupts).

Xinu loads the kernel entry point beginning at 0x8000 1000, and upon booting begins execution at that address. Within Embedded Xinu, the memory segments are in the following order (as defined by ld.script): text, read-only data, data, and block started by symbol (BSS, uninitialized data). The kernel allocates a small kernel stack after the BSS segment and finally initializes a dynamic memory heap for the remaining physical memory addresses.

The kernel memory allocator (memget and memfree), will allocate memory from the kernel heap as requested. Since Embedded Xinu uses a single page table, all kernel addresses will be mapped read-only in all address spaces, giving a user thread the ability to read from but not write to kernel memory.

Kernel Segment 1

The second kernel segment, or KSEG1, is the range of memory addresses from 0xA000 0000 through 0xBFFF FFFF. This memory unmapped and uncached, meaning that when the processor attempts to access an address in this range it will not consult the memory manager for a mapping and it will bypass the on-chip memory cache for memory loads and stores.

By skipping the hardware cache, KSEG1 will see slower memory accesses because it must get data directly from the RAM. Because of this, it is not typical to use KSEG1 for normal kernel operations, rather this segment is useful for accessing memory that is mapped to some other hardware device on the platform. These mappings will either be pre-existing, so they are out-of-range of physical memory addresses, or they will be dynamically allocated memory that will be shared between the operating system and some hardware device.

KSEG1 under Xinu

Embedded Xinu uses several hardware devices that are mapped out-of-range of physical memory and some hardware devices that use dynamically allocated memory for sharing. Some devices on the WRT54GL that are beyond the range of physical memory are:

  • Broadcom I/O controllor registers at 0xB800 0000,
  • UART registers at 0xB800 0300 and 0xB800 0400,
  • Broadcom Ethernet 47xx registers at 0xB800 1000,
  • Broadcom Wireless LAN controller registers at 0xB800 5000,
  • Broadcom 47xx RoboSwitch registers at 0xB800 6000, and
  • Flash memory (4 MB) read mapped beginning at 0xBC00 0000.

Certain drivers (such as the Ethernet driver), also take advantage of shared memory between the operating system and the hardware. This requires the use of dynamically allocated kernel memory (originating in KSEG0), that has been mapped to KSEG1 address range. This is not problematic because both KSEG0 and KSEG1 use a 1-1 memory mapping. With the Ethernet driver of Embedded Xinu, the shared memory that is in KSEG1 hold the DMA descriptor rings and the Ethernet packet buffers to store the packets in.

Kernel Segment 2

The third kernel segment, or KSEG2, is the range of memory addresses from 0xC000 0000 through 0xFFFF FFFF. This memory is both mapped and cached, meaning that the processor will consult the memory manager for a mapping and store memory modifications in the on-chip cache.

Like the user segment of memory any attempt to access memory in KSEG2 will result in the processor querying the memory manager and the TLB to find a mapping. If a mapping does not exist the processor will generate a TLB load or store exception and the operating system must fill the TLB entry. Unlike USEG, a TLB exception will not jump to the "fast" handler and instead follow the normal path for exception handling through the generic exception mechanism.

This memory segment could be useful to create the appearance of page aligned data to the underlying hardware or operating system if needed.

KSEG2 under Xinu

Embedded Xinu does not make use of any KSEG2 memory yet. However, to take advantage of the Context register of MIPS processors when a TLB exception occurs, it is possible that a mapping of the system page table to KSEG2 might exist in future versions.

References

Sweetman, Dominic. See MIPS Run. San Francisco: Morgan Kaufmann Publishers, 2007.