An udupi can mean different things to different people. A town in Karnataka, a vegetarian cuisine that first started there or a barebones no-frills cafe serving South Indian food. In Bombay though, an udupi restaurant can also be any neighborhood cafe you can eat familiar, homely food at no great expense.
But if you really want to see an udupi (the cafe, not the town), the only place to head to is Matunga in Central Mumbai. They are all the same, these cafes with metal chairs and tables, quick and efficient service and the best South Indian food on earth.
The first time I went to Matunga's King Circle, which probably has ten udupis next to each other, I settled on Cafe Mysore. It was not a choice really, for I went early in the morning and they were the only one that seemed open and in business. I've since been to Cafe Madras and Idli House and A. Ramanayak and countless others. All of these offer the snacking choices of idlis, dosas and uttapams. Some also offer the thali, depending on what time you get there.
Still I suggest you stick to Cafe Mysore, as I have done once I was finished sampling all the rest. For starters, it's bigger so you stand the best chance of getting a table (in Cafe Madras, the most popular of the lot, you will always end up sharing one). This also means that they will rush you less. Udupis in general are not places to linger - you order, you eat, you leave - but at least in Cafe Mysore they won't mind your staying an extra few mintues.
And finally the food. It's uniformly good, no matter which udupi you land at but you will notice a subtle difference in flavors everywhere. And Cafe Mysore seems to agree the most with my palate with its not-too-spicy sambar and even more amazing rasam vada. I love their idlis, but I love their masala dosa more. And I can never leave without eating the funnily named ulundu dosa, made with just the lentils and no rice. And dare you leave without sampling their filter coffee - there is no better place to go and overeat in all of Bombay.