Difference between revisions of "User:Michael"

From Embedded Xinu
Jump to navigation Jump to search
(→‎Week of June 15: More upgrades and note taking!)
(→‎Week of July 27: Merged safemem into trunk-2.0)
 
(26 intermediate revisions by the same user not shown)
Line 80: Line 80:
 
** Then proceeded to upgrade all the wikis to use a "continuous integration" process like [http://www.wikipedia.org/ Wikipedia] does, so now all the wikis should use the latest trunk version of MediaWiki.
 
** Then proceeded to upgrade all the wikis to use a "continuous integration" process like [http://www.wikipedia.org/ Wikipedia] does, so now all the wikis should use the latest trunk version of MediaWiki.
 
** Modified some settings for the SMI repository.
 
** Modified some settings for the SMI repository.
−
** Began writing a document discussing what administration tasks there are with mulug.
+
** <strike>Began writing a</strike> Completed a draft document discussing what administration tasks there are with mulug (available in <code>/var/mulug/README</code>).
 +
* Memory Protection
 +
** Began looking into system call interface.  I've decided Xinu was never meant to have user mode -> kernel mode syscalls (see 20.5 of ''Operating System Design'').  Implying that the ancient SYSCALL type is a bit of a misnomer.  Though the interface is possible (and fairly easy), it would require special wrapper functions around every call deemed to be a system call.
  
 
=== Week of June 22 ===
 
=== Week of June 22 ===
 +
* Reviewed Paper
 +
** Maeda, S., Asano, S., Shimada, T., Awazu, K., and Tago, H. 2005. ''A Real-Time Software Platform for the Cell Processor''. IEEE Micro 25, 5 (Sep. 2005), 20-29. DOI=http://dx.doi.org/10.1109/MM.2005.79
 +
* Memory Protection
 +
** Filling out TLB EntryHi and EntryLo[0,1] fields with data from processes page table.  Doesn't work.  Getting Bus error exception with both fast (assembly) and slow (C) handlers.
 +
** Building a version of [http://www.openwrt.org/ OpenWRT] to (hopefully) dump and look at values in TLB.
 +
** Fixed above problem (right shift PFN 6 bits), I thought they were 'reserved' bits---I was wrong, it works now
 +
** Fixed a few other bugs to get TLB working consistently in mapped memory space
 +
** Began working on syscall interface, still needs testing.
 +
* Got some benchmarks to run with nepsim and told [http://acm.mscs.mu.edu/wiki-reu/index.php/User:Rberg Ryan] how to build nepsim on 64-bit machines (and execute without a segfault).
 +
* Helped with some computational sciences for high school teachers activities
  
 
=== Week of June 29 ===
 
=== Week of June 29 ===
 +
* Reviewed paper
 +
** Klassner, F. and Continanza, C. 2007. ''Mindstorms without robotics: an alternative to simulations in systems courses.'' In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (Covington, Kentucky, USA, March 07 - 11, 2007). SIGCSE '07. ACM, New York, NY, 175-179. DOI=http://doi.acm.org/10.1145/1227310.1227372
 +
* Added syscall interface to safemem branch, now some cleanup work is necessary and a re-review of the files added. (+ commenting functions)
 +
* Tinkered with KVM on Fedora 11 (it seems to work well).
 +
* Helped find/fix various bugs in network stack.
  
 
=== Week of July 6 ===
 
=== Week of July 6 ===
 +
* Began working on cleanup/documentation/confirmation of all memory related subsystems (region allocator, paging, tlb, syscall interface)
 +
** Stuck thinking about how to allocate the allocators. (Region based, byte based, malloc/memget, ...)
 +
* Read
 +
** Yang, X., Cooprider, N., and Regehr, J. 2009. ''Eliminating the call stack to save RAM.'' In Proceedings of the 2009 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools For Embedded Systems (Dublin, Ireland, June 19 - 20, 2009). LCTES '09. ACM, New York, NY, 60-69. DOI=http://doi.acm.org/10.1145/1542452.1542461.
 +
* Adapted to Fedora 11 on mawdryn
 +
** Installed "morbius" virtual machine on mawdryn (doesn't have a MAC address in the database, so none of the standard configuration script ran.)
 +
* Went to Door County on Friday (sorry).
  
 
=== Week of July 13 ===
 
=== Week of July 13 ===
 +
* Worked on memory allocator
 +
** Pages are allocated to a process (but not recovered until process is killed).
 +
** memget tries to allocate from the freelist of the current process, if enough memory doesn't exist it fetches another page (or pages) and allocates it to the process' freelist.
 +
** memfree simply returns a pointer with nbytes of memory to the process' freelist.
 +
** working on some bugs with memRegionReclaim when a process is killed.
 +
** corrected bug in reclaim
 +
** completed kernel/user allocators (still needs some testing)
 +
** memRegionTransfer should also move the memlist from process to process
 +
* Read paper
 +
** Takacs, G., Chandrasekhar, V., Gelfand, N., Xiong, Y., Chen, W., Bismpigiannis, T., Grzeszczuk, R., Pulli, K., and Girod, B. 2008. ''Outdoors augmented reality on mobile phone using loxel-based visual feature organization.'' In Proceeding of the 1st ACM international Conference on Multimedia information Retrieval (Vancouver, British Columbia, Canada, October 30 - 31, 2008). MIR '08. ACM, New York, NY, 427-434. DOI=http://doi.acm.org/10.1145/1460096.1460165.
 +
* Most importantly, I updated the Xinu banner for release of 2.0.
 +
      ____  ___.__                   
 +
      \  \/  /|__| ____  __ __     
 +
        \    / |  |/    \|  |  \     
 +
        /    \ |  |  |  \  |  /     
 +
      /___/\  \|__|___|  /____/     
 +
            \_/        \/      v2.0 
 +
* Helped various people in the lab.
  
 
=== Week of July 20 ===
 
=== Week of July 20 ===
 +
* Read paper for Tuesday
 +
** Quach, B., Balakrishnan, M., Benhaddou, D., and Yuan, X. 2009. ''Implementation of integrated wireless health monitoring network''. In Proceedings of the 1st ACM international Workshop on Medical-Grade Wireless Networks (New Orleans, Louisiana, USA, May 18 - 18, 2009). WiMD '09. ACM, New York, NY, 63-68. DOI=http://doi.acm.org/10.1145/1540373.1540390
 +
* Re-implemented safemem slightly
 +
** Now has a separate user heap (using malloc/free calls) and kernel heap (using memget/memfree)
 +
** User heap is of size #define UHEAP_SIZE (currently set to 8MB) in xinu.conf
 +
*** If UHEAP_SIZE is not defined, there is no user heap and malloc always returns NULL
 +
** Only user heap memory is mapped into page table
 +
** Since Embedded Xinu uses only memory protection, we use a single global page table holding the page table entry and address space identifier (thread id)
 +
** When a TLB exception occurs, the system attempts to load the page table entry
 +
*** If the entry is null or the current thread id is not the ASID for the entry, the thread is killed and outputs "Memory protection violation" to stderr
 +
** The entire kernel is mapped as read-only, global memory so any thread can read kernel memory, but not write to it.
 +
** The TLB can be enabled/disabled by defining/undefining the USE_TLB #define in xinu.conf
 +
** Entire user memory subsystem is in the mem/ folder with only four calls in the kernel (each protected with #ifdef UHEAP_SIZE)
 +
*** system/kill.c attempts to memRegionReclaim when the thread is killed.
 +
*** system/initialize.c acquires UHEAP_SIZE using memget, then attempts to initialize the memory subsystem with memInit (setup the user heap), safeInit (setup memory protection), and safeKmapInit (insert kernel mappings).
 +
** TLB initialized with one call in system/initialize.c protected by #ifdef USE_TLB with all TLB code residing in mem/ as well.
 +
* Helped various people in the lab.
 +
* Relaxed in Northern Wisconsin Thursday and Friday
 +
** Read paper for Tuesday
 +
*** Mankin, J., Kaeli, D., and Ardini, J. 2009. ''Software transactional memory for multicore embedded systems''. In Proceedings of the 2009 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools For Embedded Systems (Dublin, Ireland, June 19 - 20, 2009). LCTES '09. ACM, New York, NY, 90-98. DOI=http://doi.acm.org/10.1145/1542452.1542465
  
 
=== Week of July 27 ===
 
=== Week of July 27 ===
 +
* Wrote catalogue of activities from last week.
 +
* Began looking at some network code
 +
** Began getting sad.
 +
* Corrected bug in tlbMissHandler that would effectively disable interrupts
 +
** Originally when a memory violation occurred the handler would kill(thrcurrent), exlset(), and resched().
 +
*** Obviously this is incorrect. kill(thrcurrent) has a call to resched() in it and exlset doesn't allow exceptions to occur.
 +
** Now when a memory violation occurs the handler will exlreset() and kill(thrcurrent).
 +
* Moved around code in malloc and memRegionAlloc to more logically separate thread based memory and region based memory.
 +
** memRegionAlloc will allocate the region and return the region allocated to the caller.
 +
** malloc tries to allocate memory from thread->memlist, then tries a memRegionAlloc and inserts remaining memory in thread->memlist
 +
* Expanded the Embedded Xinu [[memory]] page by 30%!
 +
** Fleshed out the [[flash memory]], [[TRX header]], [[flash driver]], and [[memory management]] pages.
 +
* Placed the PlayStation 3 in the rack and connected it.
 +
** Installed (pre-built) powerpc cross compiler on mawdryn, it seems to build correct binaries.
 +
* Attempted to merge the safemem branch into trunk-2.0.  Failed.
 +
** Corrected mis-allocation (skip accounting info) bug in malloc
 +
** Corrected unbalanced disable/restore bug in malloc
 +
** Wrapped memRegionFree into free(), since it does not deal with memory regions, just thread->memlist
 +
** '''Succeeded'''. [[tracsrc:mips/branches/safemem|safemem]] is now integrated in [[tracsrc:mips/branches/trunk-2.0|trunk-2.0]]
 +
*** if any bugs arise, they should be reported on trac, and disabling safe mem is fairly simple
 +
**** <code>compile/xinu.conf</code>: comment out the UHEAP_SIZE and USE_TLB #defines
 +
**** <code>compile/Makefile</code>: remove the "mem" component
 +
**** <code>lib/libxc/Makefile</code>: uncomment "CFILES += malloc.c free.c" (line 29)
 +
**** Now you should have the more "traditional" memory system ;)
  
 
=== Week of August 3 ===
 
=== Week of August 3 ===

Latest revision as of 00:49, 31 July 2009

Currently I am a in-between graduate student working at Marquette University. I have completed my thesis (Using Software Transactional Memory in Interrupt-Driven Systems), but it was about one week late — so my graduation was pushed back to August 2009. At the end of the summer I will move to St. Louis and begin working on a Ph.D. in Computer Science at Washington University.

However, until then I am a research assistant for Dr. Dennis Brylow. At present I am continuing my research with software transactional memory and operating systems. In my spare geek time, I plan on fleshing out or starting up various parts of the Embedded Xinu O/S, as well as doing various administrative lab tasks.

If you want more detailed information about me and my academic career you can visit the website of Michael J. Schultz!

Weekly Log (Summer 2009)

Week of May 11

Week of May 18

  • Discussed REU Wiki with REU students.
  • Wrote Flash memory and memory protection pages to the trac wiki with deadlines and some information.
  • Worked on KVM a bit more (no success yet) -- it seems to fail to create the graphics properly which halts the installation process.
  • Created SVN account and branch for wireless driver developer.
  • Began some preliminary work on memory protection (moving to region based memory)

Week of May 25

  • Enjoyed some relaxation time in Galena.

Week of June 1

Week of June 8

  • Got rejected.
  • Reduced conferences to look at for paper submissions.
  • Finished work on region based memory allocator. (source)
    • Xinu now (inefficiently) allocates page sized memory when requests are made.
  • Read paper
    • Stitt, G., Chaudhari, G., and Coole, J. 2008. Traversal caches: a first step towards FPGA acceleration of pointer-based data structures. In Proceedings of the 6th IEEE/ACM/IFIP international Conference on Hardware/Software Codesign and System Synthesis (Atlanta, GA, USA, October 19 - 24, 2008). CODES/ISSS '08. ACM, New York, NY, 61-66. DOI=http://doi.acm.org/10.1145/1450135.1450150
  • Built page mapping/unmapping functionality into Xinu for virtual memory
    • TODOs now include:
      • Hacking together TLB initialization and modification code.
      • Correctly use TLB during context switch.
      • Build system call interface and associated system call functions.
      • Make distinction between user-/kernel-level threads and/or processes.

Week of June 15

  • Reviewed papers
    • Ryzhyk, L., Chubb, P., Kuz, I., and Heiser, G. 2009. Dingo: taming device drivers. In Proceedings of the Fourth ACM European Conference on Computer Systems (Nuremberg, Germany, April 01 - 03, 2009). EuroSys '09. DOI=http://doi.acm.org/10.1145/1519065.1519095.
    • Fuad Tabba, Mark Moir, James R. Goodman, Andrew W. Hay, and Cong Wang. NZTM: Nonblocking Zero-indirection Transactional Memory. In Proceedings of the 2009 Symposium on Parallelism in Algorithms and Architectures (Calgary, Alberta, Canada, August 11 - 13, 2009). SPAA '09. URI=http://www.virtutech.com/files/academics/spaa091-tabba.pdf.
  • System Administration
  • Memory Protection
    • Began looking into system call interface. I've decided Xinu was never meant to have user mode -> kernel mode syscalls (see 20.5 of Operating System Design). Implying that the ancient SYSCALL type is a bit of a misnomer. Though the interface is possible (and fairly easy), it would require special wrapper functions around every call deemed to be a system call.

Week of June 22

  • Reviewed Paper
    • Maeda, S., Asano, S., Shimada, T., Awazu, K., and Tago, H. 2005. A Real-Time Software Platform for the Cell Processor. IEEE Micro 25, 5 (Sep. 2005), 20-29. DOI=http://dx.doi.org/10.1109/MM.2005.79
  • Memory Protection
    • Filling out TLB EntryHi and EntryLo[0,1] fields with data from processes page table. Doesn't work. Getting Bus error exception with both fast (assembly) and slow (C) handlers.
    • Building a version of OpenWRT to (hopefully) dump and look at values in TLB.
    • Fixed above problem (right shift PFN 6 bits), I thought they were 'reserved' bits---I was wrong, it works now
    • Fixed a few other bugs to get TLB working consistently in mapped memory space
    • Began working on syscall interface, still needs testing.
  • Got some benchmarks to run with nepsim and told Ryan how to build nepsim on 64-bit machines (and execute without a segfault).
  • Helped with some computational sciences for high school teachers activities

Week of June 29

  • Reviewed paper
    • Klassner, F. and Continanza, C. 2007. Mindstorms without robotics: an alternative to simulations in systems courses. In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (Covington, Kentucky, USA, March 07 - 11, 2007). SIGCSE '07. ACM, New York, NY, 175-179. DOI=http://doi.acm.org/10.1145/1227310.1227372
  • Added syscall interface to safemem branch, now some cleanup work is necessary and a re-review of the files added. (+ commenting functions)
  • Tinkered with KVM on Fedora 11 (it seems to work well).
  • Helped find/fix various bugs in network stack.

Week of July 6

  • Began working on cleanup/documentation/confirmation of all memory related subsystems (region allocator, paging, tlb, syscall interface)
    • Stuck thinking about how to allocate the allocators. (Region based, byte based, malloc/memget, ...)
  • Read
    • Yang, X., Cooprider, N., and Regehr, J. 2009. Eliminating the call stack to save RAM. In Proceedings of the 2009 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools For Embedded Systems (Dublin, Ireland, June 19 - 20, 2009). LCTES '09. ACM, New York, NY, 60-69. DOI=http://doi.acm.org/10.1145/1542452.1542461.
  • Adapted to Fedora 11 on mawdryn
    • Installed "morbius" virtual machine on mawdryn (doesn't have a MAC address in the database, so none of the standard configuration script ran.)
  • Went to Door County on Friday (sorry).

Week of July 13

  • Worked on memory allocator
    • Pages are allocated to a process (but not recovered until process is killed).
    • memget tries to allocate from the freelist of the current process, if enough memory doesn't exist it fetches another page (or pages) and allocates it to the process' freelist.
    • memfree simply returns a pointer with nbytes of memory to the process' freelist.
    • working on some bugs with memRegionReclaim when a process is killed.
    • corrected bug in reclaim
    • completed kernel/user allocators (still needs some testing)
    • memRegionTransfer should also move the memlist from process to process
  • Read paper
    • Takacs, G., Chandrasekhar, V., Gelfand, N., Xiong, Y., Chen, W., Bismpigiannis, T., Grzeszczuk, R., Pulli, K., and Girod, B. 2008. Outdoors augmented reality on mobile phone using loxel-based visual feature organization. In Proceeding of the 1st ACM international Conference on Multimedia information Retrieval (Vancouver, British Columbia, Canada, October 30 - 31, 2008). MIR '08. ACM, New York, NY, 427-434. DOI=http://doi.acm.org/10.1145/1460096.1460165.
  • Most importantly, I updated the Xinu banner for release of 2.0.
      ____  ___.__                    
      \   \/  /|__| ____  __ __       
       \     / |  |/    \|  |  \      
       /     \ |  |   |  \  |  /      
      /___/\  \|__|___|  /____/       
            \_/        \/       v2.0
  • Helped various people in the lab.

Week of July 20

  • Read paper for Tuesday
    • Quach, B., Balakrishnan, M., Benhaddou, D., and Yuan, X. 2009. Implementation of integrated wireless health monitoring network. In Proceedings of the 1st ACM international Workshop on Medical-Grade Wireless Networks (New Orleans, Louisiana, USA, May 18 - 18, 2009). WiMD '09. ACM, New York, NY, 63-68. DOI=http://doi.acm.org/10.1145/1540373.1540390
  • Re-implemented safemem slightly
    • Now has a separate user heap (using malloc/free calls) and kernel heap (using memget/memfree)
    • User heap is of size #define UHEAP_SIZE (currently set to 8MB) in xinu.conf
      • If UHEAP_SIZE is not defined, there is no user heap and malloc always returns NULL
    • Only user heap memory is mapped into page table
    • Since Embedded Xinu uses only memory protection, we use a single global page table holding the page table entry and address space identifier (thread id)
    • When a TLB exception occurs, the system attempts to load the page table entry
      • If the entry is null or the current thread id is not the ASID for the entry, the thread is killed and outputs "Memory protection violation" to stderr
    • The entire kernel is mapped as read-only, global memory so any thread can read kernel memory, but not write to it.
    • The TLB can be enabled/disabled by defining/undefining the USE_TLB #define in xinu.conf
    • Entire user memory subsystem is in the mem/ folder with only four calls in the kernel (each protected with #ifdef UHEAP_SIZE)
      • system/kill.c attempts to memRegionReclaim when the thread is killed.
      • system/initialize.c acquires UHEAP_SIZE using memget, then attempts to initialize the memory subsystem with memInit (setup the user heap), safeInit (setup memory protection), and safeKmapInit (insert kernel mappings).
    • TLB initialized with one call in system/initialize.c protected by #ifdef USE_TLB with all TLB code residing in mem/ as well.
  • Helped various people in the lab.
  • Relaxed in Northern Wisconsin Thursday and Friday
    • Read paper for Tuesday
      • Mankin, J., Kaeli, D., and Ardini, J. 2009. Software transactional memory for multicore embedded systems. In Proceedings of the 2009 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools For Embedded Systems (Dublin, Ireland, June 19 - 20, 2009). LCTES '09. ACM, New York, NY, 90-98. DOI=http://doi.acm.org/10.1145/1542452.1542465

Week of July 27

  • Wrote catalogue of activities from last week.
  • Began looking at some network code
    • Began getting sad.
  • Corrected bug in tlbMissHandler that would effectively disable interrupts
    • Originally when a memory violation occurred the handler would kill(thrcurrent), exlset(), and resched().
      • Obviously this is incorrect. kill(thrcurrent) has a call to resched() in it and exlset doesn't allow exceptions to occur.
    • Now when a memory violation occurs the handler will exlreset() and kill(thrcurrent).
  • Moved around code in malloc and memRegionAlloc to more logically separate thread based memory and region based memory.
    • memRegionAlloc will allocate the region and return the region allocated to the caller.
    • malloc tries to allocate memory from thread->memlist, then tries a memRegionAlloc and inserts remaining memory in thread->memlist
  • Expanded the Embedded Xinu memory page by 30%!
  • Placed the PlayStation 3 in the rack and connected it.
    • Installed (pre-built) powerpc cross compiler on mawdryn, it seems to build correct binaries.
  • Attempted to merge the safemem branch into trunk-2.0. Failed.
    • Corrected mis-allocation (skip accounting info) bug in malloc
    • Corrected unbalanced disable/restore bug in malloc
    • Wrapped memRegionFree into free(), since it does not deal with memory regions, just thread->memlist
    • Succeeded. safemem is now integrated in trunk-2.0
      • if any bugs arise, they should be reported on trac, and disabling safe mem is fairly simple
        • compile/xinu.conf: comment out the UHEAP_SIZE and USE_TLB #defines
        • compile/Makefile: remove the "mem" component
        • lib/libxc/Makefile: uncomment "CFILES += malloc.c free.c" (line 29)
        • Now you should have the more "traditional" memory system ;)

Week of August 3

Retrieved from "https://xinu.cs.mu.edu/index.php?title=User:Michael&oldid=3494"