Crawler CS188

Custom homemade replica of Berkeley's CS188 class implemented in C++ and wxWidgets by Patrick Marangone (@pmarangone) and Sergei Kononov (@hemulens). Please find our original project repository here.

Target Interface Examples

Example 1

Example 2

Tentative TODOs

Build GUI

• Create top and bottom panels
• Add two buttons: ["run", "stop"] + see other buttons needed
• Make the bottom control panel neat:
  • Step delay, discount, epsilon, learning rate
  • Create output fields / text fields with background
  • Round edges of spin controls: (canceled)
    • SetThemeEnabled
    • SetExtraStyle, SetWindowStyleFlag, SetWindowStyle
    • SetLayoutDirection, SetWindowVariant
  • Wire up controls:
    • Create and connect IDs and functions
    • Set increment step sizes
    • Set default values (through Robot)
    • Set min/max ranges
    • Read values from spin controls
    • Pass values from spin controls to Robot class instance living in the MainFrame (_robot)
• Refactor panels:
  • Move all three panels to panels.h (canceled)
  • Create two polymorph classes from wxPanel (TopPanel, BottomPanel) (canceled)
  • Set controls ownership to the parent class
  • Destroy spin controls in the parent class destructor (bottom panel) (canceled)
  • Destroy btn controls in the parent class destructor (top panel) (canceled)
  • Destroy panels in the parent class destructor
• Ownership of control variables:
  • Move '_learningRate', '_discount' and '_epsilon' to Robot
  • Transfer data to Robot class
  • Make control handles private class members
• Abstract away layout initialization process
  • Create menu initialization function
  • Create panel initialization function
  • Create app layout initialization function
• Repair the bug: clicking on the panel makes num * 1000:
  • Try the following:
    • AcceptsFocusFromKeyboard()
    • SetCanFocus(false)
    • Freeze()
    • wxTextCtrl / wxSpinCtrlDouble inherited functions:
      • SendDestroyEvent
      • OnSetFocus
      • OnKillFocus
      • OnTextLostFocus
      • OnFocus
      • DisableFocusFromKeyboard
      • OSXSimulateFocusEvents
  • Cout event name when a panel clicked after incrementing a spin control (change to wxTextCtrl happens on unfocus)

Create Graphics

• Embed a bitmap frame into the main window:
  • Implement rule of five for Graphics
  • Switch _graphics to smart pointers
• Learn wxWidgets' bitmap and rendering mechanism under the hood:
  • Consider wxDC and wxBitmap
  • IMPORTANT: Research what NOT to do and how to use (and why) wxTimer vs Render Loop
• Render a static floor
• Render crawler's body:
  • Create robot's body dimensions
  • Create robot's arm and hand

References & Resources

CS188 Reinforcement Learning resources

• Main CS188 lecture
• Auxiliary lecture resources
• The GUI of inspiration
• GitHub reference repo

wxWidgets bitmap resouces

• Drawing Sample, wxDC, wxBitmap

Temporal Difference and Q-Learning resources

• Stanford CS234: Reinforcement Learning - Model Free Control
• RL Course by David Silver - Model Free Control
• Reinforcement Learning Book (Ch. 6)
• Udacity's Deep RL - Temporal Difference (notebook)
• University of Alberta: RL Specialization