See the Revision History

Press the same button that you used to Start the recording (it's a multi-use toolbar button). Alternatively, you can use the keyboard hotkey combo, which is "Ctrl+Alt+Shift+R"  by default (can be changed in the Options). This works for both the start & stop recording. By design, this keyboard combo is omitted from the recording itself.

TinyTask is supposed to watch for the key presses Break or ScrollLock or Pause (the emergency stop keys), and immediately stop playback. Another method is the "Playback Hotkey" combo that should terminate a playback, which is "Ctrl + Shift + Alt + P" by default (note the 4 keys all held down). This key combo can be changed in the menu: Prefs > "Playback hotkey".

If it still doesn't stop, it is possible that another app has captured the keyboard or otherwise interfered. Fortunately, future versions of TinyTask (being developed in 2019) will include the ability to pick any key or combo you wish. (Note: the Escape key is not a good choice for emergency stops, since it's so commonly used for normal interactions during recordings.)

A version was created that included full customization of the Start/Stop/Play key mappings (hotkeys). Unfortunately, the extra size added to the program was unacceptable for the TinyTask concept. As a compromise, the current key combo options were chosen to avoid conflicts with other programs, while still providing some basic customization beyond a single hard-coded key mapping.
This is under consideration for a future upgrade.

UPDATE: This is currently being actively developed for 2019.

Run as Administrator ^{Source: [1]  [2]  (see comments sections for both references)}

It runs in both foreground & background, but only interacts with visible foreground apps. This is by design, since it is a GUI automation tool (it works with running applications), not a batch/script execution engine.