Minerals that are more concentrated are cheaper to extract. In general. If you think about a 40 ppm ore body versus a 2 ppm ore body, to get the same amount out of each, you have to process (mine, crush, refine) 20 times as much rock. That costs money. [This can be even more extreme if you consider yield--it is generally easier to get higher percentage extraction in a high grade ore than a low grade one...consider if your leach process has a minimum of say 0.5 ppm heading to tails...that impacts you more with 2 ppm start than with 40.]
There will be some differences Au vs Ag based on the chemistry--I think especially with competition with copper in adsorption/refining. So I'm not sure which is cheaper if equal grades (and similar host rock, but that's not given either) to extract. But just the sheer difference in concentration makes a vast difference in costs which drives up the cost of gold production.
Price has allowed a lot of gold mining to occur that would not if prices were lowered. You can process lower grade ores and use more extreme practices. I just got done with several months of operations consulting at a complicated gold processing plant. Basically throughput was main driver, yield was second (assuming grade fixed by geology). Cost was for all practical purposes irrelevant.
Sure they would do a calculation for any new project or change in operations. But pretty much anything that would increase gold ounces per year was going to carry its costs. But that works a lot better at $2000/oz vs. at $500. This leads to industry costs on overage increasing. The industry has a long history with this--if/when prices crash, you will see cost cutting that is dramatic.
P.s. It is common that gold and silver are found/extracted in similar bodies. So a gold mine/plant will have some silver as a byproduct. Sometimes this even justifies the economics for a marginal plant. However, in general, you can look at the production of these two metals relatively independently. (Copper may also be a recoverable byproduct in a gold mine, depends on the concentration...kind of a nuisance for gold processing generally but if enough of it, you can justify separating it and selling it, versus letting it go to tails.)