By default, the macOS Finder re-opens all windows from the last session when logging in.
The following LaunchAgent uses AppleScript to close all Finder windows on startup:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.example.CloseFinder.agent</string>
<key>LimitLoadToSessionType</key>
<string>Aqua</string>
<key>RunAtLoad</key>
<true/>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/osascript</string>
<string>-e</string>
<string>tell application "Finder" to close every window</string>
</array>
</dict>
</plist>
Put it into ~/Library/LaunchAgents/com.example.CloseFinder.agent.plist and load it using launchctl load ~/Library/LaunchAgents/com.example.CloseFinder.agent.plist
On the next startup, there should be no more open Finder windows from the last session.