How many is too many???

So DH and I are searching for a new kitty. Maddie just seems to be getting rather clingy and she seemed to be a happier kitty when we had Sissy too. We miss the kitties playing and racing through the house too. But there is a problem. How do you pick?? There are a LOT of kittens out there....wonderful, gorgeous, interesting, sweet ones. How many cats can you have and not be considered the crazy cat people? Because DH is just as nuts for cats or more so than I am. There are a couple kitties that he's interested in and he's started saying maybe we should get both of them.....Well I'm thinking the poop will be the deciding factor. Really now, how many litter pans are we willing to clean??