[tune] Refactor TrialRunner to separate out executor calls

[krfricke]

Why are these changes needed?

We are planning to refactor the Ray Tune trial lifecycle management by offering an improved execution engine. The TrialRunner currently encapsulates most of the execution logic. When switching the Ray execution backend, we would like to retain most of this logic. Thus, we prepare the refactor by restructuring the TrialRunner:

• The TrialRunner is split into a _TrialRunnerBase and TrialRunner class
• The _TrialRunnerBase contains all lifecycle logic that is independent from execution. For instance, processing trial results, and scheduling trial training
• The TrialRunner class contains the communication with the RayTrialExecutor
• Our future TuneController (which is prototyped) will also inherit from _TrialRunnerBase

Please note that this PR does not change any logic. Is only renames some internal methods and changes the class structure.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[xwjiang2010]

Approve to unblock the remaining one PR, since this PR is mostly splitting the logic in TrialRunner into executor-independent, and executor-dependent one (to be replaced by the new controller).

Overall feedback is that TrialRunner (minus anything searcher/scheduler related) and Trial possibly should become "AIR concept" rather than "Tune concept".