By Simon Johnson and Anna Ringstrom

STOCKHOLM (Reuters) -The head of Sweden's Moderate Party, Ulf Kristersson, said on Wednesday he would begin the work of forming a new government after Prime Minster Magdalena Andersson conceded her Social Democrats had lost the weekend's general election.

The Moderates, Sweden Democrats, Christian Democrats and Liberals appear set to get 176 seats in the 349-seat parliament to the centre-left's 173 seats, according to the latest figures from the election authority.

A handful of votes remain to be counted, but the result is unlikely to change significantly.

"I will now start the work of forming a new government that can get things done," Kristersson said in a video on his Instagram account.

The election marks a watershed in Swedish politics with the anti-immigration Sweden Democrats, a party with roots in the white supremacist fringe, on the threshold of gaining influence over government policy.

The success of the party, which took over from Kristersson's Moderates as the country's second biggest, has raised fears that Sweden's tolerant and inclusive politics are a thing of the past.

However, their mantra that Sweden's ills - particularly gang crime - are a result of decades of overgenerous immigration policies have hit home with many voters.

Kristersson said he would build a government "for all of Sweden and all citizens".

"There is a big frustration in society, a fear of the violence, concern about the economy, the world is very uncertain and the political polarisation has become far too big also in Sweden," he said. "Therefore my message is that I want to unite, not divide."

Though Kristersson's party is smaller, Sweden Democrat leader Jimmie Akesson cannot get the broad backing from the right needed to oust the Social Democrats.

Kristersson is likely to try and form a government with the Christian Democrats and rely on support in parliament from the Sweden Democrats and Liberals.

WORRY

Prime Minister Andersson accepted defeat, but warned that many Swedes were worried about the Sweden Democrats' election success.

"I see your concern and I share it", she said.

The Sweden Democrats aim to make Sweden the European Union's toughest on immigration policy including legislation making it possible to deny people seeking asylum based on religious or LGBTQ grounds.

The party wants to slash economic benefits for immigrants and give more powers to police, including zones in troubled areas allowing searches without concrete suspicion of a crime.

The Sweden Democrats look set to win 20.6% of the vote, against 19.1% for the Moderates. The Social Democrats will be at 30.4%.

Commanding only a thin majority, Kristersson faces a number of challenges, not least the fact of his party's junior status.

Forming an administration and agreeing a budget will not be easy as the Liberals and Sweden Democrats refuse to serve together - or separately - in government and differ on many policies.

"Sweden is now going to get an administration that is only one or two parliamentary seats away from a government crisis," Andersson said.

She said the her door was open to Kristersson if he wanted to rethink his alliance with the Sweden Democrats.

In addition, Sweden is in the midst of a cost-of-living crisis and could be heading for recession next year.

Russia's war in Ukraine has destabilised the Baltic region - Sweden's backyard - and uncertainty remains over whether Turkey will finally agree to Stockholm's application for NATO membership.

Measures to address climate change and long term energy policy also need to be thrashed out while holes in the welfare system exposed by the pandemic need to be plugged and a planned surge in defence spending financed.

The result still has to be officially confirmed, probably by the weekend.