Currently I think a virtual private server (VPS) solution is the best bet for those who need to start small, and while EC2 has some advantages its pricing is currently a lot higher. Going the VPS route has some scalability, some vendors (such as linode.com) offer about a 10:1 scaling ratio in features across their offered services.
Once you have maxed out a typical VPS vendor's offerings you are in the price range of a single dedicated server so the migration path could be continued by switching to a dedicated server or by getting your own hardware and perhaps co-locating it. Doing this could add about a factor of 5 to the scaling curve, so in total, the virtual and dedicated private server approaches should allow you to scale your application about 50 times without having to rework the architecture or selected technology. Once you have grown to encounter those limits you are probably leaving the domain of the startup, so its probably time for a rethink anyway.