Though I do not agree 100% with your position, you have made some good points, Ken and I think I might need to tweak my mindset.
This - "The only decision you have to make is whether you want to help the fallen climber out. " - is a good point but almost everyone's impulse is to help a victim out. Acting against this impulse might weigh heavily on one's conscience, especially if the victim dies from his/her injuries. If I had helped, might his/her life been saved?
Allow me to recount probably the 1st rescue I was involved in; a gumby huffed and puffed up the trail, no gear to speak of, no harness, asked us where the "easy" routes were. We pointed at a 5.7/5.8. He took a look and decided it was not to his liking and decided to solo an uncleaned, slime covered, broken section of the crag instead. We were dumbfounded. Sure enough, about 15 feet up, he slipped and landed loudly on his side. Much moaning ensued so we ran over. It was obvious that there was only injury to his hip and leg. And some light bleeding from scratches and such. Someone ran to get help while we stayed with him to keep him conscious and comfortable. When help arrived, we thought we were going to have a fatality, not of the soloist but one of the helpers who was panting so hard I thought he was either going to have a heart attack, pass out, or both! Long story short, injured guy was taken to the ER (I am sure he recovered from his physical wounds) and our plans for the day were done for. To this day (many, many years later), I pack up as quickly as I can and leave whenever I learn of a soloist climbing nearby. I suppose you could say I have a bad mindset but I did have a formative experience early on.