I didn't like physics to start with, actually flunked one class (I like to blame it on the theory that learning is inversely proportional to alcholic consumption...it may be logarithmic, actually, my experiment ended too soon), but then ended up getting a Bachelor's degree in astro-physics (I know, it is hard to believe that George Jetson's dog has a whole branch of pyhsics dedicated to him).
It makes it kind of tough when the guy that "wrote the book" is also the professor (Simon, physics 311, mechanics).
My favorite book of all time was actually a math book "Div, Grad, Curl and All That"....honest, that's the title!
Hang in there, it will get better. See if you can find something in the real world to apply it to or relate it to, I know calculating the time it takes for a pea to freeze (assuming an onion-layering approach and integrating across the boundaries) isn't much fun, but try relating it to something...if it doesn't get better in a year (give it at least 1 1/2 years), then bail!!!