Pre-train your representations before learning judgement

Learning how to compress the world and represent its relevant information in the most useful way is an important skill (this is the essence of Representation Learning). Learned representations are more important than what you do with them, so focus most of your time on training your representation encoder. This is how Ben Franklin’s Autoencoder worked. This will pay dividends later, because With the right representation, judgement is cheap. Judgement will come much more naturally once you’ve pre-trained.

With regards to chunking per Augmenting Long-term Memory, this means that one should learn chunks first before trying to learn judgement. For example, don’t try to learn correct chess plays directly from raw information about the board state. First, learn how to better represent the state of the board. What to do will be much more obvious once this basic information has been correctly encoded.


References

eonline