Possible issue with MC first visit algorithm

[uranio255]

Hi BY571,

I think I may have found an issue with the implementation of the following algorithm “On-policy first-visit MC control (for epsilon-soft policies)”.

In the pseudo-code version of it, we should check for the existence of St, At in past instances of time (t).
However, in the implementation we only check for the existence of St … but no At.
(image provided for clarification)

Is there a reason why you do it like this?

Thank you for any clarification!
Francisco