Plagiarism question - homework class notes?

This doesn't sounds as plagiarism at all. You said it yourself, it's an outline to take notes. Sure you might have forgotten to quote, but you are not taking away any credit from the authors of the book because you are not publishing/submitting the work as your own. This case would only be bad if you were submitting a research paper.