Kode Vicious offers a nice, short tutorial on API design in this month's ACM Queue. Getting the right balance is never easy. I face this question all the time with students in my large scale distributed computing class.

KV holds up the UNIX API set for file manipulation as the classic example of good API design:

The classic, and perhaps now cliché, example of a good API is the Unix open, close, read, write, ioctl set of system calls for performing file I/O. Unix cheated, in a way, by saying that all files were just streams of bytes without structure, but nonetheless it stands as a good example because with those five APIs most programmers can do most of the things they need to do with file I/O.

From ACM Queue - APIs with an Appetite
Referenced Thu Apr 12 2007 11:18:48 GMT-0600 (MDT)

KV goes on to talk about the importance of ioctl as an escape valve--allowing programmers to pass data down into the bowels of the OS, bypassing the API. To be more specific, the UNIX file API assumes everything thing is a stream of bytes. ioctl allows devices that are not represented by streams of bytes to be controlled in a device-specific manner.

I find that people have trouble knowing when to be impure. For example, in EJBs, the pure way to access data is through entity beans, but if all you want to do, for example, is get a list of all the records that meet a certain property so they can be displayed, using the entities is really slow. The answer is to--gasp!--just use some SQL from the session bean. Experienced EJB programmers do this all the time, but n00bs frequently dismiss it as improper.

As KV says in closing, API design is iterative. As you see people doing things with the escape valve over and over, then its probably time to provide that functionality in the API proper. That way your API will be neither too fat or too thin.