Yes - batteries can be confusing!
If I am to calculate the mAh needed for the power supply, I would just multiply the 2A current consumption by the 4 hours, right?
Yes, your calculation is correct as a first approximation. Your calculation does not account for Peukert's Law, or some of the myriad other factors that complicate battery calculations & selection. But it's a reasonable (actually a perfect) place to start. Here's the energy equation for the Raspberry Pi (aka the load in this question):
W = V x I x t = 5 volts x 2 Amps x 4 hours = 40 watt-hrs
With respect to powering an RPi, your first calculation (2 A x 4 Hrs) yielded an estimate of the load's energy requirement expressed in amp-hour terms. Because we know the RPi operates at 5V, you can "get away" with using amp-hours as a substitute for energy. But to be generally correct, you should always include voltage in your energy requirement calculations. If it helps, you can think of energy as volt-amp-hours.
Put into "correct" terminology, the two important relationships are:
Power: P = V x I
Energy: W = P x time
In other words, Power (dc power) is the product of Voltage and Current. Energy is the product of Power delivered over a period of time.
Also, I need the power supply to be 9V. Is that not considered in the calculation of the mAh at all?
I'm supplying two answers to this part of the question, as there are two different points to be made:
Answer 1:
It's all about energy. Once again: energy is power over a period of time. Bear with me as we walk through the relationships between power, energy, voltage, current, and battery capacity; I'll try to tie this all together.
Quite logically, you have set the RPi's energy load requirement as a requirement on your battery source. In other words, the source capacity must meet or exceed the load demand. Referring to the calculations above, this means you need a source/battery with a storage capacity of 40 watt-hrs or greater.
Now - let's keep in mind that (electrical) energy is fungible in the sense that volts and amps are simply two of the metrics used in the calculation. In other words, there is no energy difference in a 9V, 4A source vs. a 3V, 12A source; assuming time is the same, both provide the same amount of energy.
But of course when we set about actually selecting/building/connecting hardware, we must take the volts and amps into account - let's deal with that now. Out of necessity, I'm going to make an assumption here: When you say a "power supply", I'm assuming that you mean a battery - or some type of energy storage device. And you've said, "I need the power supply to be 9V". The schematic below illustrates this, and it adds a component necessary to power the RPi from a 9V source: a voltage regulator.
Also please note: Depending upon specifically which power supply you choose, you may - or you may not - need to add a voltage regulator. If the voltage regulator is built into your power supply, simply consider the voltage regulator and the battery as a single unit - labeled as "The Power Supply" in the schematic. The important part here is only this: The Raspberry Pi must have 5V supplied to its input.
simulate this circuit – Schematic created using CircuitLab
Answer 2:
Also, I need the power supply to be 9V. Is that not considered in the calculation of the mAh at all?
Well, yes and no. :) You see, the mAh rating is intended to convey how much energy the storage device/battery/power supply can hold. However, if a manufacturer quotes his 12V battery as having energy storage of 1000 mAh, it is implied that 1000 mAh is at 12V! Similarly, if another mfr quotes his 3.6V Li-Ion battery at 1000 mAh, it is implied that 1000 mAh is at 3.6V. And hopefully, you can now see that there is a huge difference in the actual energy stored in these two batteries due to the different battery voltages:
for the 12 V battery:
W = 12V x 1000mAh = 12,000 milliwatt-hours (12 watt-hours) of energy
for the 3.6V battery:
W = 3.6V x 1000mAh = 3,600 milliwatt-hours (3.6 watt-hours) of energy
I hope that helps.
Addendum:
We know that the RPi is quite finicky about its power; its power source must be delivered at a voltage very near 5 volts to function.
As far as the current goes, the circuitry on the RPi determines how much current it draws. We know that this current load will vary depending upon numerous factors - primarily the "computational load" we impose on the RPi through the software we run on it.
