Do Caltech's free online course "Learning from data" to learn ML in a principled way. You will get the answers to all those questions and then some. You basically need to learn some basic theory and concepts (linear models, non linear transformations, vc dimension, learning curves, regularization, validation, etc) much more than tools first to deal with those questions. The professor in the video lectures is excellent