Mr Albanese’s Labor Party defeated Scott Morrison’s conservative government in an election on Saturday.
It remains unclear whether Mr Albanese will form a majority or govern with the support of crossbenchers.
The prime minister left for Tokyo on Monday to meet the leaders of the so-called Quad nations – the US, India and Japan.
Earlier in the day, he was sworn in with four key cabinet members, including new Foreign Minister Penny Wong, who is travelling with him to Japan.
Richard Marles is the new deputy prime minister and employment minister, Jim Chalmers is treasurer, and Katy Gallagher is attorney-general and finance minister.
It is Australia’s first Labor government in almost a decade. The party has won 74 lower house seats but counting continues to determine whether they can get the 76 needed to form a majority.
But the primary vote for both major parties fell – almost a third of Australians put the Greens, independents and other minor parties as their first preference.
The Quad group is seen as largely aiming to counter growing Chinese influence in the Indo-Pacific region.
It will meet on Tuesday following recent diplomatic tensions in the Pacific, after the Solomon Islands last month signed a security pact with China.
The US and Australia hold fears the deal could allow China to build a naval base there.
In a statement ahead of the meeting, Mr Albanese said: “The Quad Leaders’ Summit brings together four leaders of great liberal democracies – Australia, Japan, India, and the United States of America – in support of a free, open and resilient Indo-Pacific.”
Ms Wong – Australia’s first overseas-born foreign minister – signalled they would bring “new energy and much more to the table” on climate action, after “a lost decade”.
Climate change played a huge role in the election result, with a surge in support for candidates wanting urgent action.
Mr Morrison’s government had committed to a 2030 emissions reduction target of 26%-28% – about half that of the UK and US. Mr Albanese’s government has a target of 43%.
he Greens are expected to pick up four lower seats, adding to seven climate-focused independents. They could put pressure on Labor to take even stronger action, especially if it fails to reach a majority.
Mr Morrison’s unpopularity and his party’s stance on climate have been blamed by some Liberal MPs for wiping out their vote.
Losses included senior party figures, including deputy leader Josh Frydenberg, in traditional Liberal strongholds.
Mr Morrison stepped down as party leader on Saturday and former defence minister Peter Dutton is the favourite to succeed him.
Mr Dutton – from the party’s right – has been a controversial figure at times. Some question whether he could rebuild Liberal support in more progressive, metropolitan areas.