As someone who used to work in the research dept. of a major media conglomerate, I can say that the networks care, at least to a degree. I used to do what they call "post-analyses," in which I'd go over the ratings for a bunch of markets, compare the figures with what we promised to deliver, and see who came out ahead. Then I'd give these figures to the sales dept. and they'd give them to the ad agency, who'd show them to the client -- and don't think that, with millions of dollars at stake, every person involved wasn't sweating a bit. I don't think either the network, the ad agency, or the client REALLY know how many people are watching -- Nielsen, or Arbitron in radio, is imperfect. But they can break down the demographic detail pretty well, and if they feel they aren't hitting the kind of audience they want, they'll pull their funds -- and at a few hundred grand per 30-second spot, that's a lot of money. Which hits the networks where they live.
What clients are after is exposure, and if you fast-forward through their ad on the VCR, you still see it. Moreover, ad agencies and clients are probably as concerned with "clutter" -- their ad gets lost in the mass of commercials -- as with channel-surfers and tape-spinners. Hell, it's all a crap shoot, but where else can you even attempt to hit 30 million people at once?
Well, I've rambled, but you get the idea.