Teaching 4-5 courses per semester is a lot of work, and places where 4-5 courses per semester is the norm tend to have larger class sizes, making it even more work. And, yes, even at places where research is not a priority, some scholarly work is expected for tenure and promotion, and of course meetings, committee work, and dealing with students out of classroom and office hour time is a lot of work.
Being a tenured faculty member at a decent university is a fairly cushy gig, but a) it takes a hell of a lot of work to get there and b) that describes a pretty small proportion of people known as 'college professors.' I worked all the time when I had a tenure track job, even though some friends and relatives thought I only worked 6 hours per week, as that's the amount of time I spent in the classroom. Even once I opted out of the research game and spent a couple of years doing a cushy teaching gig with few other demands on my time (unlike most people), I probably put in a normal 40 hour week.
