I know the minimum required rate is 20KB, and the minimum rate to be considered "fast" is around 100KB, but even the "fast" rate seems incredibly slow to me, compared to the average download speed of a residential broadband connection in the US (~4 MB/s).
Is there a rough consensus around a recommended minimum rate that meets the expectations of the average Tor user?
There's basically always a shortage of bandwidth in the Tor network, so every amount of Bandwidth you can donate helps.
Additionally, your relay usually serves more than one circuit at a time, so any one connection that's going through your relay will receive only a share of your bandwidth anyway.
The amount of circuits goes up the more bandwidth you donate, so it doesn't really matter for the individual connections, as long as you have some minimum, but the more bandwidth you donate, the faster the network will become overall.
Keep in mind that your relay won't be fully utilized from the start. The network needs time to "get to know" your relay. Check out this blog-post about the life-cycle of a new relay.
The Tor Project now recommends at least 250 kilobytes/second each way to keep the level of service at least somewhat sane for users. Providing less bandwidth than that is actually a net drain on resources inside the network, because information about all relays has to be transmitted to all clients, which adds up to surprising amounts of bandwidth. Also, it's plainly no fun to be on a relay that can't even serve a youtube video, and there is very little a relay with slow upload can do.
It is a battle of privacy vs performance, see https://blog.torproject.org/lifecycle-of-a-new-relay/#comment-34637
