Today, I got an "401: Unauthorized" exception when calling the WebClient.DownloadFile method to download an Infopath file from SharePoint, even if I provided the correct permissions (setting the WebClient.Credentials property).
This was because of the Infopath Forms Server feature in MOSS 2007 Enterprise edition. This feature redirects to the /_layouts/FormServer.aspx page, apparently this was resulting in an Unauthorized exception.
A workaround is to add "?NoRedirect=true" to the url, like this: http://ServerName/sites/SiteCollection/FormLibrary/Form1.xml?NoRedirect=true.
More information about some Query Parameters you can use on the /_layouts/FormServer.aspx page: http://msdn.microsoft.com/en-us/library/ms772417.aspx