QRW07 Cross validation for speed-bonus (L,E) take 2

Problem QRW6: Results still too unstable.

Goal: Find better values for L and E based on the results of QRW06

QRW06 shows the best results for L0 and E0. These were the highest values for these parameters. For this training we try those values and higher values.

epoch count: 10k

training.simrunner.RewardHandlerName.speed-bonus

Considers all simulation events for calculating the reward.

Possible simulation events created for an agent:

1. After every simulation step:

   • Pushed the opponent. Reward = +0.05
   • Is pushed by the opponent. Reward = -0.01

2. At simulation end:

   • Winner by pushing the opponent: Reward = 100 + t * 150
   • Looser without being pushed: Reward = -100 - t * 150
   • Looser being pushed: Reward: -10

(t * x) is the 'speed bonus'

t = 1 - (s / max_s)

s:     Number of steps when th simulation ended   
max_s: Max number of steps for a simulation

Means, the reward/penalty is higher the shorter the simulation ran. The agent gets a higher reward when fast pushing out the opponent, or a higher penalty when fast moving unforced out of the field.

training.parallel.ParallelConfig.q-rw-3

L0 E0 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E0 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E0 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E1 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E1 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E1 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E2 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E2 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L0 E2 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E0 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E0 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E0 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E1 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E1 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E1 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E2 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E2 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L1 E2 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E0 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E0 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E0 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E1 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E1 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E1 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E2 D0 M0 R0

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E2 D0 M0 R1

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10

L2 E2 D0 M0 R2

q-values
video 0 video 1 video 2 video 3
video 4 video 5 video 6 video 7
video 8 video 9 video 10