Login and Download File Programmatically in .NET

Recently, I was working in a .NET project when I needed to login to a website and download a file programmatically. I was able to call the login page & set the cookies without any problems. However, when I tried to download the file, I got weird messages like “Object moved to…” or “Unable to read data from the transport connection: The connection was closed”. The messages varied depending on the AllowAutoRedirect property and a few others.

I have spent hours to solve the problem, searched MSDN and dozens of forums without any success. Then, it turned out to be something really stupid. The thing is; when you are accessing the page to download the data, you have to pass some HttpWebRequest parameters; such as the user agent, protocol version and accept.

You will find the working code below. The critical area is marked bold.

public string readData()
{
CookieContainer cc = new CookieContainer();
HttpWebRequest request;
HttpWebResponse response;
CookieContainer cookies;
WebProxy proxy;
string ret = “”;

// Prepare proxy
proxy = new WebProxy(“192.168.0.10:8080”, true);

// Login & get cookies
request = (HttpWebRequest)WebRequest.Create(@”https://www.dummy.com/login.aspx?userid=myuser&password=mypass”);
request.AllowAutoRedirect = false;
request.CookieContainer = new CookieContainer();
request.Proxy = proxy;
response = (HttpWebResponse)request.GetResponse();
cookies = request.CookieContainer;
response.Close();

// Get XML file
request = (HttpWebRequest)WebRequest.Create(@”https://www.dummy.com/download.aspx?CMD=XML”);
request.AllowAutoRedirect = true;
request.CookieContainer = cookies;
request.Proxy = proxy;
request.AllowAutoRedirect = true;

request.Method = “GET”;
request.ProtocolVersion = new Version(1, 0);
request.UserAgent = “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)”;
request.Accept = “*.*”;

response = (HttpWebResponse)request.GetResponse();
ret += new StreamReader(response.GetResponseStream()).ReadToEnd();
response.Close();

// Logout
HttpWebRequest req3 = (HttpWebRequest)WebRequest.Create(@”https://www.dummy.com/logout.aspx”);
req3.CookieContainer = cc;
req3.Proxy = proxy;
HttpWebResponse res3 = (HttpWebResponse)req3.GetResponse();
res3.Close();

// Return values
return ret;
}

Advertisement