Answers to this question that concentrate on the personalities of the people involved at top are not really historical nor true. To describe Trotsky “as a great ideas man but easily bored“ or as “ annoying “ is laughable. This was the man who planned the execution of the October Revolution and then built and headed the Red Army To victory over the Whites and their 14 western allies.
The revolution in Russia was always regarded by the Bolsheviks at that moment as nothing more than a stepping stone. Lenin reflected this view by his frank assessment in 1920 that the Soviet Union was capitalist state with communists in charge. Unless revolution became international, the rocky alliance between two antagonistic classes, workers and peasants would fail. The state he noted was actually administered by the Tsarist bureaucracy.
Not only that, by 1921, the working class who had carried through the revolution had virtually ceased to exist. The only way forward in the wake of this was NEP, as a holding policy to enable Russia to recover by allowing the capitalist market, regulated carefully, to resume to encourage the peasantry to grow and market food. A holding policy because the key to the revolution’s success lay not in Russia but in the capitalist world, initially Germany.
However, with absence of a vibrant working class, to sustain it , the Bolshevik Party itself was decaying. Some old Bolsheviks, particularly Stalin, who had played an insignificant role in the revolution and Zinoviev and Kamenev who had actually opposed the October revolution were tending towards an unmarxist position that socialism could be achieved within the Soviet Union. A socialist island in a capitalist world.
Great Russian bully is how Lenin described Stalin over the “ Georgian affair”. So a rift was developing. The Party took over the state in the name of the working class. Now the Tsarist bureaucracy was slowly taking over the Party.
It is worth recalling that in July 1917, the Bolsheviks had to share support In Petrograd, with another powerful left organisation – the Mezrayhontsy, the Inter District organisation. Not a catchy name, but in it were Trotsky,Rykov, Rakovsky, Lunacharsky, the great economist Preboensky, Radek and many others. It was Lenin who went in person to negotiate with this group on an amalgamation with the Bolsheviks. Most of these dominated the Central Committee after the union of the two parties was agreed.
Moreover after the revolution the Mezrayhontsy took all the major positions in the first Council of People’s Commissars – the government. Stalin, a mediocre and unnoticed figure was given the Workers and Peasants Inspection. It was through this channel and via his party position of general secretary that state bureaucrats with no Marxist understanding or background were recruited in ever greater numbers into the Communist Party. These people would have a nationalistic outlook one which Stalin himself was heading. Whereas in 1917, the Bolsheviks headed by Lenin and Trotsky and their left supporters could always call on popular support against the conservatism of the Old Boldheviks.
In 1926 when the struggle between left and right of the party was played out, the Left now joined by Zinoviev and Kamenev found that there was no mass working class who could support them. The Soviets themselves, in 1917 and 1918 vibrant organs of working class democracy were by 1923 empty shells filled with placemen and party hacks. The workers were either dead in the war, become party men or had become apathetic.
Before his final stroke, in 1922 Lenin had recognised how bad things had got and planned with Trotsky an assault on the Party bureaucracy in the 1922 Congress. A few day before it he suffered a stroke that paralysed him.
The Joint Opposition in 1926 laid out an extremely coherent plan ( Zinoviev and Kamenev had joined Trotsky) countering the horrific tendency towards the nonsensical concept of “ socialism in one country”. They failed as they were bound to. The Soviet population was politically supine. The peasants had their land, that’s all they wanted, the organised working class had gone. Who was to prevent Stalin using the bureaucratic mechanism of state control to defeat the Opposition.
The tragedy is, these great, humanitarian socialist thinkers Radek, Rykov, Rakovsky, Yoffe, Lunacharsky met their end at the hands of anti socialist bureaucratic police regime headed by Stalin. Only Trotsky survived, too big to kill, because in the popular mind at that time he was still closely associated with Lenin and the revolution.
The Bolsheviks failed and the Soviet Union collapsed into a murderous state capitalist bureaucratic nightmare because their principal failure was the failure of the revolution in Europe. The idea that the Soviet Union proves that Communism doesn’t work is nonsense, it has yet to be tried. The Russian revolution was the first step, but capitalism proved to be too strong. It won’t always be the case.