Are you happy living in Bilston? “Why wouldn’t I be?” giggles a young mother pushing a stroller through a pub car park on the Stowlawn Estate.

My question is not as daft as it sounds. Seventy-one years ago, this corner of the Black Country town was part of a pioneering social experiment, devised by exiled Viennese philosopher Otto Neurath and a visionary town council, to “make Bilston happy”.

Today, the happiness industry is a lucrative business. In recent years we’ve had a government happiness tsar, and bookshelves groaning with bestsellers including Stumbling on Happiness, The Secrets of Happiness, and The Pursuit of Happiness.

But Bilston got the jump on them all in 1945 when it invited Neurath, then lecturing in Oxford, to become its consultant on human happiness. The hope was he could bring a visionary approach to housing to Bilston, as he had done to the rebuilding of post-war Vienna.

Heaven knows, Bilston needed something to cheer it up. “Surely no place could have grown up as ugly as this without some evil mind having deliberately planned to wipe out every trace of beauty,” wrote socialist politician Fenner Brockway about this town of more than 31,000 people in 1932, in his book Hungry England.

Brockway was visiting the Black Country town, scarred by years of coal mining and pollution from blast furnaces and factories, to see its notorious slums. “I find it difficult to describe the first house I visit. It would be more suitable for chickens than human beings ... There are some streets which show efforts to maintain respectability and an appearance of comfort; but most folk seem to have given in. Poverty has got on top of them.”

Ten years later, despite building new council estates, Bilston still led the way in Britain for all the wrong reasons. A 1943 civic survey found that 1,400 tons of smoke particles per square mile were falling on Bilston each year, which local historian Frank Sharman colourfully suggests is “roughly the weight of 11 blue whales or 233 elephants per annum”.

According to Sharman, a national survey of housing conditions assessed the number of families in each local authority area who were living in “overcrowded conditions”. “The figure for England and Wales as whole was 3.8%. In Bilston it was 13.7%.” A later civic survey, just before the end of the war, found that of 7,771 houses in Bilston, 2,655 were unfit for human habitation.

Bilston, then, needed thousands more new houses for slum dwellers – and one of the prime sites for building those new council houses was land in the north-west of the borough. Today Stowlawn is green and pleasant, an estate of whitewashed houses, verdant verges and bucolic trees. But in the late 1940s, it was land which had been plundered for coal and road stone and then abandoned. “It was not exactly a greenfield site,” says Sharman. “Even calling it ‘brownfield’ would have been a bit optimistic.”

This plan to rebuild Bilston and rehouse its slum dwellers was fairly well advanced when in early 1945, Bilston’s town clerk, AV Williams, had an epiphany in Wolverhampton. He was there to attend a lecture given to the International Friendship League; the speaker was Otto Neurath, and his subject was “Housing in Vienna”.

Williams was impressed by the socialist philosopher’s humane vision: “He made one believe in the dignity of human beings.” Soon after, Neurath was invited to Bilston to see the nature of its housing problem and help eradicate it. “He made us feel that the pursuit of beauty and happiness could be achieved by the common man,” Williams explained.

Neurath believed that the common man – and woman – could and should have a role in designing their own homes. It was a perspective he had become committed to in the aftermath of the first world war in Austria, when he helped revolutionise working-class housing in the era of Red Vienna – the nickname of the capital between 1918 and 1934 when, following the collapse of the Habsburg Empire, its city government was ruled by social democrats.

It was a time and place when acute housing shortages had led to conditions of inhuman squalor – conditions, it’s not fanciful to argue, that applied to inter-war Bilston too.

During this time, Vienna became a crucible for communal living. Architecturally, the era is synonymous with vast apartment blocks such as the Karl Marx Hof and the Rabenhof, featuring communal facilities ranging from bathhouses, laundries, kindergartens, libraries and even kitchens.

Neurath was gaining fame at the time as a member of the Vienna Circle of “logical positivist” philosophers, and for developing the pictographic language isotype, one designed to help grasp by means of pictures complex information.

He was an unusual socialist, improbably influenced by Samuel Smiles, the Scottish Victorian reformer whose 1859 book Self-Help appealed to Neurath’s sensibility. Smiles had argued that poverty was caused largely by irresponsible habits and that human progress, perhaps even happiness, would come from the poor being given what Smiles called “the means of education, and of exerting freely all the powers of his godlike nature” .

This philosophy – the Christian sense of mission notwithstanding – inspired Neurath. He thought happiness could be achieved if workers could be educated on the principles of house design and communal living, and then be involved in the creation of their living spaces rather than having it inflicted on them from above – and he tried to put these ideas into practice in Vienna.

Shortly before this Jewish thinker was compelled to emigrate as Austrian fascists came to power in 1934, he saw the opening of the Werkbundsliedung, an estate of 70 houses in south-west Vienna that, as Dr Sabrina Rahman, co-curator of a recent exhibition called Bilston’s Happy Housing explains, “offered working-class families the opportunity to purchase or rent homes designed by a diverse group of 31 international architects”.

According to Rahman, “the Werkbundsiedlung remains one of the most important estates in modern architecture”. And the hope was Neurath could realise something similar in England’s Black Country.

Before Neurath came to Bilston, British architect Sir Charles Reilly was already sketching plans for Stowlawn estate, using ideas the Viennese would have found sympathetic, if a little more patrician. As Frank Sharman points out: “The chief feature of Reilly’s plans was that the houses should be built around greens, somewhat in the style of a village green, which Reilly seems to have seen as an archetype of a good community. And on each of these greens there would be a club, containing library, reading rooms, billiards room and catering facilities – to which families could go to eat or from which they could order meals to be delivered.”

These ideas chimed not only with Neurath’s but with the new Labour government’s housing minister Aneurin Bevan’s vision of a New Jerusalem in Britain – of mutually supportive, mixed communities. Bevan wrote in his introduction to the 1949 Housing Act: “We should try to introduce in our modern towns and villages what was always the lovely feature of English and Welsh villages, where the doctor, the grocer, the butcher and the farm labourer all lived in the same street.”

Neurath went further in his philosophy of how Bilston’s new estates could create such communities and make the town happy. Some were noted down by AV Williams from meetings with Neurath at the time, and are cited by Frank Sharman. Even now they seem wonderfully sane and pertinent ideas for how we should – but often don’t – live:

“On estates,” advised Neurath, “mix up individuals: married and unmarried, old and young. Do not create ghettos of, for example, old people who, if they are stuck altogether in flats, will feel isolated, lonely and unwanted. If you put them in with young people, they can do things like babysitting and feel useful and wanted.

“Unmarried persons can be divided into those who will eventually marry and those who never will. If you put them all into one block of flats, the marrying kind will move out and you will be left with nothing but people who will never marry. Care must be taken to avoid the accidental creation of a lunatic asylum.

“You must study the existing community structure in the slums you are rehousing, and ensure these connections are not broken upon the new estate. Bear in mind that different people had different needs, and do not plan housing on a ‘one size fits all’ basis. You have to work out what sorts of houses and flats to build and who to put in them.”

One big idea Neurath was keen on was an exhibition of the plans for the benefit of those who would be living in these new houses. “You must consult the people who were to be re-housed and educate them on housing matters,” he told Bilston’s town clerk and his colleagues. “You should do this through an exhibition which made the issue understandable by anyone – through the use of isotypes – and that such an exhibition should be somewhere which was accessible to the people who were to be re-housed.” Nowadays, perhaps, this sounds like common sense; in 1945 it was a radical overturning of a patrician social order.

But before Otto Neurath could make Bilston happy, he died unexpectedly aged just 63 in his Oxford home in December 1945. His widow Marie recalled that he was reading a newspaper article about happiness before he passed away, and then commented on what he had read to her.

“What does happiness mean?” he reportedly said. “I can tell you what I mean by happiness. If godfather would come to me and say: ‘Dear Otto, I make you an offer as you are living now with your books and your work and your wife. You can live for ever and ever but you will never be more prosperous than you are now. Will you accept?’ ‘Yes, dear godfather. I gladly accept.’”

After his death, Marie went back to Bilston to fulfil her husband’s happy dream. She set up his planned exhibition in a disused store on Bilston’s Oxford Street, aimed at getting slum dwellers and other ordinary citizens to see what was being planned.

The year after Neurath’s death, Ella Briggs, who had worked with Otto in 1920s Vienna, became one of the main architects of the Stowlawn estate. She intended to build eight-house terrace blocks around common greens as her predecessor Reilly had envisaged, keeping designs simple to reduce building costs in the post-war austerity era. Bilston council gave the resultant streets rustic names such as Lawnside, Fairway, and Oaklands – evoking a bucolic mood in much the same way earlier and snootier developments such as Hampstead Garden Suburb did in north London.

‘It is a happy place’

What remains of the original vision for Stowlawn today? When I stroll around the estate, I notice many of the houses remain white-painted, which was unusual for 1940s British council estates and gave this corner of Bilston a continental, crypto-Bauhaus vibe. For a while, Stowlawn was known as the White City.

Stowlawn in 2016 is undeniably peaceful and green, and no doubt a nice place to live, but it’s also a traditional English bodge – a history of compromises, frustrations and muddling through inscribed on every brick, every blade of grass.

I look in vain for the community clubs Reilly envisaged – they were never built. Many of Stowlawn’s greens were replaced with new houses when land was in short supply in the 1960s. Nowadays only Lawnside Green remains, and even it is planted densely with shrubbery and bears a sign saying “No Ball Games” – which, I suspect, Neurath would have frowned at.

Much of the original vision for the estate got dismissed by central government or obliterated by later developments, argues Frank Sharman. He then points out a particularly poignant irony in Stowlawn’s evolution.

In 1979, Margaret Thatcher was elected prime minister, and soon after embarked on a sell-off of Britain’s council estates, as part of her policy of reducing the public sector and giving the market free reign. Sharman notes that she used to hand out copies of the Viennese economist Friedrich Hayek’s The Road to Serfdom which, she said, set out her economic vision – one of minimal state intervention and negligible public ownership.

“Such ideas were, of course, anathema to Otto Neurath, who had spent much time advocating the necessity of government regulation of the free market and a degree of central planning,” writes Sharman in his superb history of the Stowlawn estate. “In fact, it seems that Hayek’s book had been written with the specific intention of refuting Neurath. And on Stowlawn the game played out – the planned community lost out to the forces of the unbridled free market.”

Did Bilston ever get happy? Well, one thing worth saying is that it was never quite as unremittingly miserable as Fenner Brockway portrayed it. My late mother went to Bilston Girls’ High School, on the southern edge of Stownlawn while the estate was being built in the late 1940s. It was a neo-Queen Anne-style building that had opened in 1929 and is now converted into luxury flats. That school, perhaps, was emblematic of the more genteel Bilston that existed alongside apocalyptic slums and industrial hell.

And my late father used to work at Quasi-Arc, the British Oxygen Corporation’s electrical welding division, in Bilston in the 1960s. That business, like many other of the Black Country’s heavy industries, is long gone. It wasn’t just the Thatcherite sell-off of council houses that changed Bilston; it was also her ideological commitment to destroying the industries that toxically thrived here.

In 1979, emblematically, the year Thatcher was elected was the year the final steel cast was made at the Elisabeth furnace at Bilston’s Spring Vale steelworks. The entrancing flames from that furnace, that I used to savour as they lit the Black Country at night from my nan’s back step in Wednesbury, were gone forever. Never such squalor, nor such grandeur, again.

These days, I reflect, as I walk from Stowlawn back to the tram stop to take me to Birmingham, I’ll bet the smoke particles falling on Bilston amount to rather fewer than 233 elephants per annum. This is, in many ways, not so much the Black Country as the Green Country. Bilston, town of furnaces, flames and inhumane squalor, once looked like Mordor; now it’s more like Hobbiton.

Otto Neurath may not have succeeded in making Bilston happy, and the ideas he set out may only have been realised in botched form – and even then diluted further. But perhaps it got there anyway. His vision of encouraging people to participate in the creation of a happier Bilston didn’t die with him.

Online, there’s a fascinating interview from 1975 with a Bilston town planner on local news programme ATV Today. With a superbly straight face, this nameless civic champion suggests putting tin foil over the gasometer so that it interestingly refracts the sunlight, pumping out coloured smoke rings from disused factory chimneys, covering an abandoned gantry with wire netting to make the Black Country’s biggest aviary, transforming the abandoned railway line into a green corridor complete with picnic areas, fishing holes and cycle paths. It’s not just Viennese philosophers who have visionary ideas for urban living.

Better yet, the interviewee invites town citizens to submit their own ideas to make Bilston beautiful. Otto Neurath would surely have liked that: anything to get locals involved in planning the circumstances of their own happiness.

True, Neurath’s vision of cities composed of mutually supportive, mixed communities, living together in harmony isn’t exactly voguish in our era of gated communities, sheltered accommodation, streets too dangerous for kids to play in. But it would be a fool or a snob who said Bilston isn’t a happy place.

“I’ve lived here all my life and I want my daughter to be as happy here as I’ve been,” says the young mother outside the pub which, I belatedly notice, is called The Happy Wanderer. “It is a happy place.”

Stuart Jeffries’s book Grand Hotel Abyss: The Lives of the Frankfurt School will be published by Verso in September

Follow Guardian Cities on Twitter and Facebook and join the discussion