Without getting into the physics of steam vs. diesel, there are several indisputable reasons for the disappearance of steam:
1. As Crandell stated, at one time, GM knew how to run a business. With all the steam engines worn out from hard service in WWII, the railroads had to order lots of new motive power. GM offered very attractive terms to railroads that wanted to buy F units. At the same time, GM was destroying the remaining streetcar and interurban lines with the same tactic through the National City Bus Lines, which GM owned.
2. MU Control. You could have a bunch of smaller diesels pull a big train with one engineer and fireman. You needed either one really big steam engine or a bunch of smaller steam engines to pull a big train and every locomotive required an engineer and fireman.
3. Infrastructure. It was expensive to haul, process, and deliver coal to steam engines. The average railroad spent almost 10% of its revenue in 1945 hauling the fuel they needed for steam engines, which was a dead loss. Steam engines needed lots of water so thousands of water facilities had to be maintained. Even railroads like the UP, that were almost 100% oil-fired, still had to maintain water facilities.
4. Standardization. An F unit was an F unit. All the important mechanical parts were identical from engine to engine so they could be maintained at any engine facility. Steam locomotives were rarely standardized even within the same class on the same railroad, so expensive machine shops had to be maintained to custom make replacement parts. Given the different requirements for fuel and water among railroads, it was almost impossible to interchange engines. With the arrival of the F and GP units, this not only became possible but common. One of the reasons F-M failed as a diesel manufacturer was sticking to the air throttle, which meant F-M units could only be used with other F-M units. GM got this early on and were successful because they built each engine much the same as they built Chevy's.
5. Pollution. Before 1950, there were few pollution laws outside New York City. LA was fighting smog and those smoky steam engines were easy to blame. Laws were being passed all over the country limiting the use of steam engines in cities, leaving the railroads to face maintaining steam power for mainline hauls and diesel power for use in cities. The economics of using one kind of engine began to make more and more sense. One of the reasons that steam hung on as long as it did in Russia, China, and India is the lack of pollution laws.
So, FWIW, those are my top 5 reasons why the diesel won and steam lost.