When I am doing the challenges should I use libraries, or write my own code? How should I determine the cut off? In some cases it is clear (e.g. if it is about learning Bootstrap I obviously need to use Bootstrap), but in others it is not (e.g. when writing the calculator app should I use something like Math.js, or write the code on my own)?
You can do whatever you want. Writing everything from scratch will probably be more difficult, but also provide you a greater learning experience. I personally recommend doing it without libraries the first time and then go back once you understand what is really needed for the project and select libraries which will reduce the amount of code you write. Everyone has their own approach and opinion about how to do the projects. This is merely my approach.