IIS - This configuration section cannot be used at this path. This happens when the section is locked at a parent level.

After installing IIS and .net frameworks I hosted applications on the IIS.However I started getting these errors when I accessed the sites. Below is the error and the resolutions.

Fix is to enable everything under Application Development of IIS installation.

Moving IIS Web Server folders to different drive.

I received new web servers that had very less disk space on c drive.If you install IIS web server,by default the web server folders i.e. inetpub will be located on the systemdrive,usually c drive. We had application logs that would require large space.In order to ensure IIS worked with folders on non system drive we had to do the following steps after installation

• Disable the Default Web Site in inetmgr
• Move the inetpub folder to D drive.You will require Admin Permissions on the server.
• Click on Default Web Site Basic Settings in inetmgr
• Point the physical path to the new folder on the non system drive.In this case it is D drive.
• Restart the Default Web Site in inetmgr

IIS - Automatically Restart stopped AppPools

I have applications that are hosted across 5 load balanced web servers with app pools for each site on the web servers. Recently I noticed that few App Pools had stopped and I had to log on to the server and restart the App pools.

One setting that helps in handling stopped App pools is the Start Mode setting.By default the value is OnDemand.By changing this default setting to AlwaysRunning, the app pool stops can be handled automatically.

c# - List all dates in a year that are a palindrome

I got a text message forward that today 5 October 2015, is a palindrome date. If you consider day first format (ddMMyyyy) then this would be 5102015.So I wanted to see if there are other dates in the year that are a palindrome.

In US the date format is always month first so for this program I am using the MMddyyyy format.So Sunday,May 10 2015 is a Palindrome.

using System;

namespace DatePalindrome
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                DateTime oFirstDay = new DateTime(DateTime.Now.Year, 1, 1);
                DateTime oLastDay = new DateTime(DateTime.Now.Year, 12, 31);
                DateTime oDay = oFirstDay;
                while (oDay < oLastDay.AddDays(1))
                {
                    string sDay = oDay.ToString("MMddyyyy").StartsWith("0") ? oDay.ToString("MMddyyyy").Substring(1, 7) : oDay.ToString("MMddyyyy");
                    
                    char [] sArray = sDay.ToCharArray();
                    Array.Reverse(sArray);

                    if (sDay.Equals(new string(sArray)))
                        Console.WriteLine("Palindrome:{0} - Day:{1}",sDay,oDay.ToLongDateString());

                    oDay = oDay.AddDays(1);
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                Console.WriteLine(ex.StackTrace);
            }
            Console.WriteLine("Done! Press any key to exit");
            Console.Read();
        }
    }
}

Now if you change the date format to MMddyy then all days from May 10 2015 to May 19 2015 are palindromes.

By making the code change for date format using this

string sDay = oDay.ToString("MMddyy").StartsWith("0") ? oDay.ToString("MMddyy").Substring(1, 5) : oDay.ToString("MMddyy");

We get

sql - Side by side counts from multiple tables

I had to create a daily report with counts from different tables in SQL Server.The report had counts for total rows in tables for a given day.Below is the sql to get counts from 7 different tables and display the counts in separate columns for a given day in the month.

select a.create_dt,con,bro,[set],conf,demo,vip,dev
from 
(select create_dt,count( id) con,row_number() over (order by create_dt) as row_num 
from table1 with (nolock)
group by create_dt
) a 
full outer join
(select create_dt,count( id) demo,row_number() over (order by create_dt) as row_num
from table2 with (nolock)
group by create_dt
) b
on a.row_num = b.row_num
full outer join
(select create_dt,count( id) bro,row_number() over (order by create_dt) as row_num
from table3 with (nolock)
group by create_dt
) c
on b.row_num = c.row_num
full outer join
(select create_dt,count(id) [set],row_number() over (order by create_dt) as row_num
from table4 with (nolock)
group by create_dt
) d
on c.row_num = d.row_num
full outer join
(select create_dt,count(id) vip,row_number() over (order by create_dt) as row_num
from table5 with (nolock)
group by create_dt
) e
on d.row_num = e.row_num
full outer join
(select create_dt,count(id) dev,row_number() over (order by create_dt) as row_num
from table6 with (nolock)
group by create_dt
) f
on e.row_num = f.row_num
full outer join
(select convert(varchar(8),create_ts,112) create_dt,count(id) conf,row_number() over (order by convert(varchar(8),create_ts,112)) as row_num
from table7 with (nolock)
group by convert(varchar(8),create_ts,112)
) g
on f.row_num = g.row_num
order by a.create_dt

.NET 4.5 Framework Initialization Error

Today while trying to run an application on a server I got the below error.

By default Visual Studio 2012 uses .Net 4.5 for as the version for build and release of projects. Even though I changed the version to .Net 4.0 and rebuilt the application and copied to the server I still was getting the error.

The reason I was still getting this error was due to the runtime support still referring to .Net 4.5 framework in the App.config file.

After changing the runtime to .Net 4.0 the application ran without any error.

Other solution is to download  .Net 4.5. 

c# - Hashset vs List performance

In one of my recent tasks I had to work with text files and extract rows based on certain values contained in a specific column.

I had a 4 GB file and I had to parse that file to extract rows that matched certain keywords.The total keyword matches varied from 10000 to less than a million.

For a smaller set of keywords I have used List but noticed that as the list items grew in size the lookup  operation became slower.Below is a test I did to compare the lookup performance between a List<string> and HashSet<string>

HashSet<string> lookup is faster and should be used for lookup instead of a list.

using System;

using System.Collections.Generic;

using System.Diagnostics;

namespace HashSetListCompare

{

    class Program

    {

        static void Main(string[] args)

        {

            try

            {

                string s = "512367";

                List<string> sNumbersList = new List<string>();

                HashSet<string> sNumbersHashSet = new HashSet<string>();

                //List load performance

                Stopwatch oListLoad = new Stopwatch();

                oListLoad.Start();

                for (int i = 0; i < 1000000; i++)

                    sNumbersList.Add(i.ToString());

                oListLoad.Stop();

                Console.WriteLine("List load time:" + oListLoad.Elapsed);

                //Hash Set load performance

                Stopwatch oHashSetLoad = new Stopwatch();

                oHashSetLoad.Start();

                for (int i = 0; i < 1000000; i++)

                    sNumbersHashSet.Add(i.ToString());

                oHashSetLoad.Stop();

                Console.WriteLine("Hash load time:" + oHashSetLoad.Elapsed);

                //List lookup performance

                Stopwatch oListWatch = new Stopwatch();

                oListWatch.Start();

                if(sNumbersList.Contains(s))

                    Console.Write("List lookup time: ");

                oListWatch.Stop();

                Console.WriteLine(oListWatch.Elapsed);

                //Hash set lookup performance

                Stopwatch oHashSetWatch = new Stopwatch();

                oHashSetWatch.Start();

                if(sNumbersHashSet.Contains(s))

                    Console.Write("Hash lookup time: ");

                oHashSetWatch.Stop();

                Console.WriteLine(oHashSetWatch.Elapsed);

            }

            catch (Exception ex)

            {

                Console.WriteLine(ex.Message);

                Console.WriteLine(ex.StackTrace);

            }

            Console.WriteLine("Done");

            Console.Read();

        }

    }

}

Hadoop - Decompress gz files from reducer output

Due to space constraints on our Hadoop clusters we had store output files in compressed gz format.

STORE finaldata INTO '/output/${month}/${folder}.gz/' USING PigStorage('\t'); 

Once we started storing our output files in the compressed format, we had to change the way we merge the output files to extract it to the local system.The getmerge command copied the files in compressed format. 

hadoop fs -getmerge /output/${month}/${folder}.gz/part-* /output/handoff.txt

In order to decompress the files and copy it over to local system we had to use the text command..

hadoop fs -text /output/${month}/${folder}.gz/part-* > /output/handoff.txt

pig - Extract data from large Bags after GROUP BY

I have multiple files with same columns and I wanted to aggregate the values in two columns using SUM.

The column structure is below

ID first_count second_count name desc
1  10          10           A    A_Desc
1  25          45           A    A_Desc
1  30          25           A    A_Desc
2  20          20           B    B_Desc
2  40          10           B    B_Desc

In wanted the below output

ID first_count second_count name desc
1  65          80           A    A_Desc
2  60          30           B    B_Desc

Below is the script I wrote .

A = LOAD '/output/*/part*' AS (id:chararray,first_count:int,second_count:int,name:chararray,desc:chararray);
B = GROUP A BY id;

C = FOREACH B GENERATE group as id,
              SUM(A.first_count) as first_count,
              SUM(A.second_count) as second_count,
              A.name as name,
              A.desc as desc;


This resulted in the the below output.The output has multiple tuples for each group i.e. the name and desc were repeated.

1  65          80           {(A)(A)(A)}{(A_Desc)(A_Desc)( A_Desc)}
2  60          30           {(B)(B)} {(B_Desc)( B_Desc)}

When I ran this script on the entire dataset that has 500 millions rows in it the reducer got stuck since the Bags after the group had large number of tuples.

In order to get the desired output.I had to distinct the tuple i.e. name and desc and then FLATTEN them to get the desired output.

D = FOREACH C 
    {
distinctnamebag = DISTINCT name;
distinctdescbag = DISTINCT desc;
GENERATE id,first_count,second_count,flatten(distinctnamebag) as name, flatten(distinctdescbag) as desc;
    }

Now the output looks like this

1  65          80          A     A_Desc
2  60          30          B     B_Desc

Increase process priority in Windows Server 2008

If you have used the task scheduler in windows server 2008 to schedule a windows task,you will see that by default the task scheduler runs the process with a Below Normal priority.
When I moved the tasks from windows server 2003 to windows server 2008, I noticed a sharp decline in performance.Most of the tasks that I have scheduled are multi threaded and IO intensive.

Even though the task settings is set to run under an Admin account with highest privileges the process runs with a Below Normal priority.

Note:You can change the priority here but I did not notice any change in the performance.

One way to bump up the process priority is in code
System.Diagnostics.Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.High
However I did not see any improvement in process performance.

After playing with the settings I was able to tweak the configuration file for the task in the task scheduler to set the process priority to High.The process now executes faster and I did see performance improvement over the previous process execution time.

Below are the screenshots of how to override windows server task scheduler process priority.

Create the task in the task scheduler.Once the task is created,right click on the task and choose export.This exports the task properties/settings to a xml file.Save the xml file on the local machine.Delete the task that was created.

Open the xml file which has the task settings and you will see the process priority set to 7.

Edit the priority and set it the desired priority.4 is for Normal and 1 is for High.Note that setting a higher priority is not recommended.My process is highly IO intensive and hence I have used  1.

Save the xml file.Go to the task scheduler,right click in the task list window and select import.Select the xml file for the process and the task should be created with a higher process priority.

Now my process runs with a High priority.Also the process performance has been higher. Even though the recommended priority is Normal, after 3 months of execution I have not encountered any issues with the process or the server.