I think you're right, but the problem is that the addition of more focused boards simply creates disparity within 4ch itself, rather than bringing in others.
Let's say, theoretically, there was an advertising effort for 4ch, paid for solely through donations, or a movement of bloggers finding refuge in 4ch. The audience would widen, and would thus grow as people would have more reason to continue returning. I think the question should be first how to raise traffic, then how to make more focused boards, etc.