The Communist Party of India was founded in Tashkent on October 17, 1920, soon after the Second Congress of the Communist International. The founding members of the party were M.N. Roy, Evelina Trench Roy (Roy's wife), Abani Mukherji, Rosa Fitingof (Abani's wife), Mohammad Ali (Ahmed Hasan), Mohammad Shafiq Siddiqui and M.P.B.T. Acharya.

The CPI began efforts to build a party organisation inside India. Roy made contacts with Anushilan and Jugantar groups in Bengal. Small communist groups were formed in Bengal (led by Muzaffar Ahmed), Bombay (led by S.A. Dange), Madras (led by Singaravelu Chettiar), United Provinces (led by Shaukat Usmani) and Punjab (led by Ghulam Hussain). However, only Usmani became a CPI party member.

During the 1920s and beginning of 1930s the party was badly organized, and in practice there were several communist groups working with limited national coordination. The British colonial authorities had banned all communist activity, which made the task of building a united party very difficult. Between 1921 and 1924 there were four conspiracy trials against the communist movement; First Peshawar Conspiracy Case, Second Peshawar Conspiracy Case, Moscow Conspiracy Case and the Cawnpore Bolshevik Conspiracy Case. In the first three cases, Russian-trained muhajir communists were put on trial. However, the Cawnpore trial had more political impact. On March 17, 1924, M.N. Roy, S.A. Dange, Muzaffar Ahmed, Nalini Gupta, Shaukat Usmani, Singaravelu Chettiar, Ghulam Hussain and R.C. Sharma were charged, in Cawnpore (now spelt Kanpur) Bolshevik Conspiracy case. The specific charge was that they as communists were seeking "to deprive the King Emperor of his sovereignty of British India, by complete separation of India from imperialistic Britain by a violent revolution." Pages of newspapers daily splashed sensational communist plans and people for the first time learned such a large scale about communism and its doctrines and the aims of the Communist International in India.

Singaravelu Chettiar was released on account of illness. M.N. Roy was in Germany and R.C. Sharma in French Pondicherry, and therefore could not be arrested. Ghulam Hussain confessed that he had received money from the Russians in Kabul and was pardoned. Muzaffar Ahmed, Nalini Gupta, Shaukat Usmani and Dange were sentenced for various terms of imprisonment. This case was responsible for actively introducing communism to a larger Indian audience. Dange was released from prison in 1925.

On December 25, 1925 a communist conference was organized in Kanpur. Colonial authorities estimated that 500 persons took part in the conference. The conference was convened by a man called Satyabhakta. At the conference Satyabhakta argued for a 'national communism' and against subordination under Comintern. Being outvoted by the other delegates, Satyabhakta left both the conference venue in protest. The conference adopted the name 'Communist Party of India'. Groups such as LKPH dissolved into the unified CPI.[5] The émigré CPI, which probably had little organic character anyway, was effectively substituted by the organization now operating inside India.

Soon after the 1926 conference of the Workers and Peasants Party of Bengal, the underground CPI directed its members to join the provincial Workers and Peasants Parties. All open communist activities were carried out through Workers and Peasants Parties.

The sixth congress of the Communist International met in 1928. In 1927 the Kuomintang had turned on the Chinese communists, which led to a review of the policy on forming alliances with the national bourgeoisie in the colonial countries. The Colonial theses of the 6th Comintern congress called upon the Indian communists to combat the 'national-reformist leaders' and to 'unmask the national reformism of the Indian National Congress and oppose all phrases of the Swarajists, Gandhists, etc. about passive resistance'. The congress did however some differentiation between the character of the Chinese Kuomintang and the Indian Swarajist Party, considering the latter as neither a reliable ally nor a direct enemy. The congress called on the Indian communists to utilize the contradictions between the national bourgeoisie and the British imperialists.[8] The congress also denounced the WPP. The Tenth Plenum of the Executive Committee of the Communist International, July 3, 1929 – July 19, 1929, directed the Indian communists to break with WPP. When the communists deserted it, the WPP fell apart