QRW07 Cross validation for speed-bonus (L,E) take 2

Problem QRW6: Results still too unstable.

Goal: Find better values for L and E based on the results of QRW06

QRW06 shows the best results for L0 and E0. These were the highest values for these parameters. For this training we try those values and higher values.

epoch count: 10k

training.simrunner.RewardHandlerName.speed-bonus

Considers all simulation events for calculating the reward.

Possible simulation events created for an agent:

1. After every simulation step:

   • Pushed the opponent. Reward = +0.05
   • Is pushed by the opponent. Reward = -0.01

2. At simulation end:

   • Winner by pushing the opponent: Reward = 100 + t * 150
   • Looser without being pushed: Reward = -100 - t * 150
   • Looser being pushed: Reward: -10

(t * x) is the 'speed bonus'

t = 1 - (s / max_s)

s:     Number of steps when th simulation ended   
max_s: Max number of steps for a simulation

Means, the reward/penalty is higher the shorter the simulation ran. The agent gets a higher reward when fast pushing out the opponent, or a higher penalty when fast moving unforced out of the field.

training.parallel.ParallelConfig.q-rw-3

Results for: QRW07 L0E0D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E0D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E0D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E1D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E1D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E1D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E2D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E2D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L0E2D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E0D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E0D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E0D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E1D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E1D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E1D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E2D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E2D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L1E2D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E0D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E0D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E0D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E1D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E1D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E1D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E2D0M0R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E2D0M0R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

Results for: QRW07 L2E2D0M0R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10