Here are some preliminary results for Small Scale Combat using Monte Cargo and MDPs.

• MC with no transition table
• MC with transition table
• MC with MDP

TODO:

• longer trials
• longer playouts

• scenario: 3v3 melee
• speed: 30
• skip speed: 100
• sample size: 3
• Num playouts: 100
• Playout Depth: 8
• neighborhood size: 7
• Playout Config
  • move chance: 0.2
  • attack chance: 0.75
  • same attack chance: 0.5
• Objective function
  • Player unit life bonus: 1.2
  • Player life factor: 0.75
  • Enenmy unit life bonus: 0.2
  • Enemy life factor: 0.75
• Table indexing
  • 3×3 table, 4^9 entries, 0=no unit, 1=obstruction, 2=player unit, 3=enemy unit

MC approach with no learning. The average should stabilize, due to a lack of learning.

This experiment uses a table that uses the most recently executed action in the action, if the table contains an entry for the index. It looks like the performance begins to improve at the end of the trial, need to run more simulations.

Using a randomized approach based on the frequency of actions appears to reduce the performance of the agent.