Depending on how many sites your Windows web server is hosting maintaining IIS logs can be a challenge. IIS logs provide valuable insight into the traffic your sites are experiencing as well as detailed SEO metrics and performance data. A typical web server will have just enough free disk space for future growth needs but ultimately will be limited by the capacity of drives in the server. If left unmonitored IIS logs can quickly fill any remaining disk space on the server. There are a few 3rd party tools that are good at compressing log files when they are under one parent folder but when the log files are in different locations such as on a WebsitePanel server I support an alternative solution is needed. In this walkthrough I will demonstrate how I solved this challenge using asp.net and GZipStream.\r\n\r\n \r\n
What about Windows Folder Compression?
\r\nEnabling Windows folder compression should always be the first step to try when freeing up space used by IIS logs. I always redirect the site logs to a folder at the root level of the server so it’s easy to find them and monitor their growth. In the example below I saved 8.3 GB of disk space without having even having to zip any files. However Windows folder compression was not going to work for my WebsitePanel server.\r\n\r\n
\r\n\r\n \r\n
Finding WebsitePanel IIS Logs
\r\nWebsitePanel is a wonderful Windows control panel for Multi-Tennant web hosting and the best part is that it’s free. If you haven’t discovered this incredible application yet check it out. Once installed every aspect of site creation and maintenance is completely automated. As shown in the picture below customer sites are usually deployed with a directory structure like c:\HostingSpaces\customername\sitename. The IIS logs are always stored further down the tree in …\sitename\logs\W3SVCXX. As one can imagine on a web server with hundreds of sites it would be quite a challenge to track down and manually apply Windows compression on all the W3SVC folders. This would be further complicated as new sites are added to the server.\r\n\r\n
\r\n\r\n \r\n
Using Directory.EnumerateFiles and GZipStream
\r\nI knew I was going to do some coding to solve my challenge but I also wanted to keep the solution easy to use. In years past I would have considered VBScript folder recursion and executing the PKZip command line tool but I wanted something more contemporary. So I used Visual Studio 2013 and created a Console Application.\r\n\r\n
\r\n\r\n \r\n\r\n
\r\n\r\n \r\n\r\nThe GZipStream class which was introduced in .Net 2.0 would handle the necessary file compression. Since I would only be compressing individual files I didn’t have to deal with using a separate library such as DotNetZip.\r\n
|
|
<span class="kwrd">public</span> <span class="kwrd">static</span> <span class="kwrd">void</span> Compress(FileInfo fileToCompress)\r\n{\r\n <span class="kwrd">using</span> (FileStream originalFileStream = fileToCompress.OpenRead())\r\n {\r\n <span class="kwrd">if</span> ((File.GetAttributes(fileToCompress.FullName) & \r\n FileAttributes.Hidden) != FileAttributes.Hidden & fileToCompress.Extension != <span class="str">".gz"</span>)\r\n {\r\n <span class="kwrd">using</span> (FileStream compressedFileStream = File.Create(fileToCompress.FullName + <span class="str">".gz"</span>))\r\n {\r\n <span class="kwrd">using</span> (GZipStream compressionStream = <span class="kwrd">new</span> GZipStream(compressedFileStream, CompressionMode.Compress))\r\n {\r\n originalFileStream.CopyTo(compressionStream);\r\n Console.WriteLine(<span class="str">"Compressed {0} from {1} to {2} bytes."</span>,\r\n fileToCompress.Name, fileToCompress.Length.ToString(), compressedFileStream.Length.ToString());\r\n }\r\n }\r\n }\r\n }\r\n} |
\r\n
The Directory.EnumerateFiles method introduced in .Net 4 would use find all the IIS log folders from the WebsitePanel hosting spaces directory tree. Using the searchOption overload would enable me to not only search for .log files but also limit the searching to specific folders matching “W3SVC”. In the snippet below I loop through the search results, compress any log file older than 1 day, and delete the original file. The last step is to just write the name the file that was just zipped to an audit log.\r\n
|
|
<span class="kwrd">public</span> <span class="kwrd">static</span> <span class="kwrd">void</span> zipFiles()\r\n{\r\n <span class="kwrd">try</span>\r\n {\r\n <span class="kwrd">foreach</span> (<span class="kwrd">string</span> file <span class="kwrd">in</span> Directory.EnumerateFiles(homeDirectory, <span class="str">"*.log"</span>, \r\n SearchOption.AllDirectories).Where(d => d.Contains(<span class="str">"W3SVC"</span>)).ToArray())\r\n {\r\n FileInfo fi = <span class="kwrd">new</span> FileInfo(file);\r\n <span class="kwrd">if</span> (fi.LastWriteTime < DateTime.Now.AddDays(-1))\r\n {\r\n Compress(fi);\r\n fi.Delete();\r\n activityLog(<span class="str">"Zipping: "</span> + file);\r\n }\r\n }\r\n }\r\n <span class="kwrd">catch</span> (UnauthorizedAccessException UAEx)\r\n {\r\n activityLog(UAEx.Message);\r\n }\r\n <span class="kwrd">catch</span> (PathTooLongException PathEx)\r\n {\r\n activityLog(PathEx.Message);\r\n }\r\n} |
\r\n
I only needed 30 days worth of log files to be left on the server. In the snippet below I delete any zipped log files matching my date criteria and then write the activity to the audit log.\r\n
\r\n
|
|
<span class="kwrd">public</span> <span class="kwrd">static</span> <span class="kwrd">void</span> deleteFiles()\r\n{\r\n <span class="kwrd">try</span>\r\n {\r\n <span class="kwrd">foreach</span> (<span class="kwrd">string</span> file <span class="kwrd">in</span> Directory.EnumerateFiles(homeDirectory, <span class="str">"*.gz"</span>, \r\n SearchOption.AllDirectories).Where(d => d.Contains(<span class="str">"W3SVC"</span>)).ToArray())\r\n {\r\n FileInfo fi = <span class="kwrd">new</span> FileInfo(file);\r\n <span class="kwrd">if</span> (fi.LastWriteTime < DateTime.Now.AddDays(-daysToSave))\r\n {\r\n fi.Delete();\r\n activityLog(<span class="str">"Deleting:"</span> + file);\r\n }\r\n\r\n }\r\n }\r\n <span class="kwrd">catch</span> (UnauthorizedAccessException UAEx)\r\n {\r\n activityLog(UAEx.Message);\r\n }\r\n <span class="kwrd">catch</span> (PathTooLongException PathEx)\r\n {\r\n activityLog(PathEx.Message);\r\n }\r\n} |
\r\n
\r\n
Using an XML Config File
\r\nTo help keep my application flexible I created a simple XML file to store the path to the WebsitePanel hosting spaces folder and the number of days of logs I wanted to keep on the server.\r\n
|
|
<?xml version=<span class="str">"1.0"</span> encoding=<span class="str">"utf-8"</span>?>\r\n<MyConfig>\r\n <HomeDirectory>D:\HostingSpaces</HomeDirectory>\r\n <DaysToSave>30</DaysToSave>\r\n</MyConfig> |
\r\n
\r\n\r\nLoading these values into my program is done by parsing the XML data using XmlDocument.\r\n
|
|
<span class="kwrd">public</span> <span class="kwrd">static</span> <span class="kwrd">void</span> readConfig()\r\n{\r\n\r\n <span class="kwrd">string</span> path = System.IO.Path.GetFullPath(<span class="str">@"logCleanConfig.xml"</span>);\r\n XmlDocument xmlDoc = <span class="kwrd">new</span> XmlDocument();\r\n xmlDoc.Load(path);\r\n XmlNodeList HomeDirectory = xmlDoc.GetElementsByTagName(<span class="str">"HomeDirectory"</span>);\r\n homeDirectory = HomeDirectory[0].InnerXml;\r\n\r\n XmlNodeList DaysToSave = xmlDoc.GetElementsByTagName(<span class="str">"DaysToSave"</span>);\r\n daysToSave = Convert.ToInt32(DaysToSave[0].InnerXml);\r\n\r\n} |
\r\n
\r\n
Running the Program
\r\n
Running the program from a command line I can see the progress as files are compressed. Afterwards I setup a scheduled task so it will run every day to compress new IIS log files and purge the ones older than 30 days.\r\n\r\n
\r\n\r\n \r\n\r\nA few days later I came across several sites on the server using CMS products with their own application logs. These log files were not being cleaned up by the site owners and taking up free space on the server so I decided to archive them as well. Since they were being stored within each site in subfolder called “Logs”, tweaking the searchOption parameters of the Directory.EnumerateFiles method would catch them. With a quick rebuild of the program, it would now compress the CMS log files as well as the IIS log files.\r\n
|
|
<span class="kwrd">foreach</span> (<span class="kwrd">string</span> file <span class="kwrd">in</span> Directory.EnumerateFiles(homeDirectory, <span class="str">"*.log"</span>,\r\n SearchOption.AllDirectories).Where(d => d.Contains(<span class="str">"W3SVC"</span>) || d.Contains(<span class="str">"Logs"</span>)).ToArray()) |
\r\n
\r\n
\r\n
In Summary
\r\n
IIS logs contain wealth of data about web site traffic and performance. Left unchecked over time the logs will accumulate and take up disk space on the server. Using Windows folder compression is a simple solution to reclaim valuable disk space. If your log files are different locations then leveraging the power of ASP.NET with Directory.EnumerateFiles and GZipStream will do the trick. Thanks for reading.
Creative, customer focused, results oriented, Senior Web Systems Engineer who enjoys providing the highest level of customer service supporting complex Windows hosting solutions. MCITP, MCSA, MCTS
More Posts - Website
. It turned out that because the site’s certificate had been revoked, browsers weren’t actually loading the site which in turn meant the redirect wasn’t happening. There was only one way to solve this last piece of the puzzle and that meant putting in a valid SSL certificate again. So I created a Certificate Signing Request for my friend’s site and within minutes they had a new $9 
