I think that Marx, and then more so Engels and the mainstream workers movement theorists that followed, mudded the waters a good deal between 'the transition to communism', 'the lower phase of communism' and 'communism'.
I think that Marx was fairly clear here. He didn't even mention the 'lower-higher' distinction much, and when he did he was clear that it was a phase of communism in which commodity production had ceased to exist, as well as bringing up similar ideas in first illustrating socialist society in volume 1 of Capital, as well as the 'no more money than a theatre ticket' comment and such. In speaking of proletarian political rule, he brought up things such as the ten planks, and on the same lines the 'transitory measures' mentioned in the IWA report on inheritance and the demands of the French Workers' Party, which presupposed commodity production and capitalism. As he stated in the well-known letter on what he had discovered, the dictatorship of the proletariat constitutes the transition to socialism. He certainly never distinguished between 'communism' and 'the lower phase of communism', the latter being of course a phase of communism, and indeed he used the latter in illustrating socialism.
We can see here the origins, in theory at least, of subsequent state capitalist and workers self management proposals being sold as socialism or communism - some honestly and some not so honestly.
I don't think that Proudhon's ideas may be put down entirely to the influence of Marx.
Can comment on articles and discussions
I think that Marx, and then more so Engels and the mainstream workers movement theorists that followed, mudded the waters a good deal between 'the transition to communism', 'the lower phase of communism' and 'communism'.
In Marx's time, in what I referred to earlier in shorthand as a 'non-revolutionary' period, a period in which capital ruled for the most part only 'formally' and also far from 'globally', it was indeed difficult to imagine how there could be such a rapid and revolutionary shift from one mode of production and social organisation to another. Various schemas were put forward which, in retrospect, we can see enabled the continuation of quite basic features of an exchange economy and which also envisaged retention of much the same forms of labour and technology into the first stages of a new society. There is also, as seen in one of Graco's quotes from Marx above, the, perhaps understandable, assumption that there is need for a whole further period of industrial expansion in a form only partially disimilar from capitalism (continued work on creating the pre-conditions for communism not fulfilled by capitalism??). We can see here the origins, in theory at least, of subsequent state capitalist and workers self management proposals being sold as socialism or communism - some honestly and some not so honestly. But capitalism (and our understanding of technology and ecology) has moved on since those days and we can now abandon those notions of what amounted in practice to 'transitional societies' and look forward to a far more rapid process of change where any hangovers from capitalism are precisely in the superstructural and ideological sphere and not in the base sphere of the mode of production.
As to capitalism creating the material 'pre-conditions' for communism and the conundrum of how change is to happen 'consciously' without there being a change on the economic mode of production beforehand to be reflected in a changed consciousness, the point goes back to the dynamic and contradictory nature of capitalism which distinguishes it from previous class societies. A society which has to expand and change through the combined and interelated process of competition between conglomerations of capital and the class struggle between workers and capitalists. A contradictory process which creates fissures within it's social and economic fabric that allow through the practical experience of struggle, particularly in periods of severe crisis, both an awareness of the impossibillity of capitalism fulfilling human needs and also the potential for revolutionary change ( though inevitably starting with minorities who need to be active in the course of those struggles).There may be more than one set of ruling ideas, but not all ideas are the ruling ideas - though distinguishing which is which is not always easy it's true.
There is however nothing predetermined or guaranteed that this process will result in communism. The problem being that working class struggle is periodic, so that short of revolution, it will also play a part in modernising capitalism rather than undermining it and thus allowing it further leases of life. On top of that minority organisations of pro-revolutionaries are not immune to the 'ruling ideas' in any period and even more so in prolonged periods of low levels of working class struggle. The risk here is that even when struggle increases such organisations ( in their ideas, and in their organisational structure and practice) may be ill equiped to play a genuinely revolutionary role and may even play (as they have in the past) a reactionary one. In todays tiny and dispersed pro-revolutionary movement this is a particuar problem which I have addressed on other threads previously.
I've struggled to express myself well here but hopefully some points are a bit clearer than they were?