A Biologist's guide to mathematical modeling in evolution and ecology (Otto) is a very good book that is presented for people that have a highschool (or sligthly higher) level in mathematics. It makes a good review about all subjects that are usually taught to first year students in Biology such as linear algebra for example. It is highly accessible and in the meantime it goes pretty far as it ends up talking about the application of diffusion equation in population genetics (Kimura). This book presents some important models in population genetics but as it aims to provide the tools for mathematical modeling in ecology and evolution it may under-considerate some fields of ecology and evolutionary biology. The book does not talk about population structure nor about evolutionary game theory or Coalescent theory for example.
There are several books that offer an introduction to population genetics. I read Principles of Population Genetics (Hartl & Clarke). I appreciated it but if I were you I think I would rather try Elements of population genetics (Charlesworth) or Population Genetics (Hamilton). There is also Genetics of Populations (Hedrick). I would tend to think that this last book presents lots of empirical population genetics data and doesn't take as much focus as the others in theoretical concepts (but I might be wrong).
There's a book called Mathematical population genetics (Ewens). I am currently reading it. It is a good book but it is definitely not an introductory book and it really doesn't cover much of the most common fields in population genetics.
If you are particularly interested in age-structured population, Evolution in age-structured populations (Charlesworth) is a very good book. Coalescent theory offers a very interesting set of mathematical tools in evolutionary biology.
Coalescent Theory: an Introduction (John Wakeley) is a good book. I haven't read it completely for both time issues and because the math are a bit complicated for me.
Modeling evolution (Roff) offers some discussion on how to mathematically define fitness from phenotypic traits. While it is interesting I would not counsel you to buy it. Moreover, all the mathematics are quite basic and it aims to explain how to perform mathematical modeling with R which is to my opinion not essential to learn as other software make a better job at dealing with math (Mathematica for example).
Evolutionary Conservation Biology (Ferrière, Couvet and Dieckmann) is a very good book of conservation and conservation genetics. It develop some mathematical models that are of special interest to conservation of populations and communities.
Ecology, Genetics and Evolution in Metapopulations (Hanski and Giaggiotti) is a book that may interest you as well. However it focuses much more on ecology than the other ones I cited above. Note: I haven't read it entirely.
So in short, I'd recommend Sally Otto's book if you want to ensure your knowledge in mathematics. Then, you should try one of the book I suggested in the second paragraph but as I know only one of them, you might want to wait other answers to get different opinions.
Hope that helped!