Valheim is a new Early Access game on Steam.
You can set up a dedicated server using steamcmd and appid 896660
Forward TCP and UDP ports 2456-2458 to your server if you're behind NAT or whatever.
Now for the real reason for this post.
Joining a server directly isn't as easy as you'd think.
But it does integrate with the steam server stuff that I always forget about.
From the Steam client, click on the "View" menu at the top and choose "Servers"

Click on the "Favorites" tab

Click on "ADD A SERVER" at the lower right

Type in the hostname (or IP) and port
ADD THIS ADDRESS TO FAVORITES

Click "Refresh" on your favorites tab

Select the new server and "CONNECT"
You may have to do this a few times, something is weird

Enter the password and click "CONNECT"

The game will launch, choose your character and all that. Enter the password again and you're good to go.
I hope this process improves a bit in the future