java - BitConverter

I am currently converting one of the C# .Net program into a pig script that uses a java UDF.The program processes a million+ gz files into tab delimited text files. 
One of the functions in the C# .Net program is to compute the MD5 hash of a few fields in the gz file and convert the hash to a long.I did not find an equivalent function for BitConverter in Java.

Below is the C# code.

 private static long ComputeHash(string p)
 {
            MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
            byte[] hash = md5.ComputeHash(Encoding.ASCII.GetBytes(p));
            return BitConverter.ToInt64(hash, 0);
 }

The Java equivalent is below

private long ComputeHash(String sHashString) 
{

MessageDigest md5 = MessageDigest.getInstance("MD5");

byte [] hash = md5.digest(sHashString.getBytes());

ByteBuffer buffer = ByteBuffer.wrap(hash);

buffer.order(ByteOrder.LITTLE_ENDIAN);

return buffer.getLong();

}

Below are the test results of Java program and C# .Net.The input is a column from the gz file and output is the string and its equivalent MD5 hash converted to long.Note that results on the left are from the PIG script that is using the Java UDF and the results on the right are from C# .Net program.

The remote server returned an error: (400) Bad Request.

In one of the projects that I was working on I started getting this error as soon as the site was hosted on the test server.
To give a brief background on the requirement, I had a site that was making a HttpWebRequest to the home page of another site to get the cookies.These cookies were then reused in subsequent requests.The sample code which was working on my dev machine is below

HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(new Uri(sSiteHomeUrl));
oRequest.Method = "GET";
oRequest.ContentType = @"application/x-www-form-urlencoded";
oRequest.CookieContainer = oCookies;
oRequest.Accept = @"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
oRequest.UserAgent = @"CHROME browser version: 33.0 user agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36";
oRequest.Headers["Accept-Encoding"] = @"gzip,deflate,sdch";
oRequest.Headers["Accept-Language"] = @"en-US,en;q=0.8,de;q=0.6,es;q=0.4";

string sResponseData = string.Empty;          
using(WebResponse oWebResponse = oRequest.GetResponse())
using (Stream data = oWebResponse.GetResponseStream())
using (StreamReader reader = new StreamReader(data))
{
      sResponseData = reader.ReadToEnd();
}

Once I tested it on my dev machine I moved the code and hosted the application on the test server.However on the test server I started getting this error.After a day of struggling and going through various forums I was able to fix the issue.

The problem was that both my site that was making a call to another site were on the same server under the same IIS. When creating the URL I was using the server name in the url.By replacing the servername with the Ip address I was able to successfully make the request from my site and get the cookies for subsequent requests.

Old call
http://servername:12348/Home.aspx

New call
http://ipaddressofserver:12348/Home.aspx

c# - Get all attribute values that exceeds int32 max value in an xml

I maintain an application that collects millions of xml files.I have parsers that then run to extract the information from these xml files.Recently we noticed that there was a drop in the number of xml files we were receiving.
We started seeing this error in the logs that caused the drop in the xml posts

System.OverflowException: Value was either too large or too small for an Int32.

So we grabbed one of the xml files that was failing and looked for any attribute that was causing the xml post to fail.The xml file size varies on the data and it was hard to figure out which attribute was causing the failure.
In order to figure this out I wrote a small program to list all the attributes that had a value which exceeded the Int32 value.Using this we were able to identify the data limits and fix it. This code can be tweaked to check for boolean,long,date,string size etc. Below is the code

using System;
using System.Xml.Linq;

namespace GetInvalidXmlIntElement
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                var oXmlDoc = XDocument.Load(@"c:\Data\Sample.xml");
                var oElements = oXmlDoc.Descendants();
                var iElementsCounter = 0;
                var iAttributeCounter = 0;
                var iInvalidValueCounter = 0;
                foreach (var oElement in oElements.Elements())
                {
                    iElementsCounter++;
                    foreach (var oAttribute in oElement.Attributes())
                    {
                        iAttributeCounter++;
                        try
                        {
                            if (Convert.ToInt64(oAttribute.Value) > Int32.MaxValue)
                                Console.WriteLine(" {0}) Element {1} Attribute {2} Int32 Max {3} Value {4} ",++iInvalidValueCounter,oElement.Name, oAttribute.Name, Int32.MaxValue, Convert.ToInt64(oAttribute.Value));
                        }
                        catch { }
                    }
                }
                Console.WriteLine();
                Console.WriteLine("Total Number of Elements in the Xml is:{0}", iElementsCounter);
                Console.WriteLine("Total Number of Attributes in the Xml is:{0}", iAttributeCounter);

            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            Console.Read();
        }
    }
}

Task Scheduler 0x1

In one of the projects I am working on,I had to create a few scheduled tasks in the Task Scheduler.The server is running on Windows Server 2008 OS.I created the tasks but when it ran I saw the following 0x1 status in the last run result column of the task scheduler.

Neither the event viewer showed any error nor the history tab for the tasks.The solution was that I had not set the working directory for the task.After setting the working directory the tasks ran without any issues.

See below screenshots

Hope that helps

c# - Delete browser cookies,cache

 Directory.Delete(Environment.GetFolderPath(Environment.SpecialFolder.InternetCache),true);

UPDATE failed because the following SET options have incorrect settings: 'ARITHABORT'

While testing one of the web applications I built, I started getting this error on one of the stored procedures.The application was using 3 databases to manage data and the users would make changes to a local database and after reviewing the changes would move the changes to the staging and then an admin would move all the changes by the users to the final database.So,In my original test application setup,I had 3 databases i.e. local,staging,final on the same server and the stored procedure to update the staging from local database was working.

However when we moved the test database to a different server and modified the stored procedure to point to the new test server we received this error.

UPDATE failed because the following SET options have incorrect settings: 'ARITHABORT'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations.

The fix was to reconfigure the user options on the database that was moved to the new server

sp_configure 'user options', 64
reconfigure

More about the user options here http://technet.microsoft.com/en-us/library/ms190763.aspx

asp.net - Disable ASP.NET_sessionId

In one of the applications I developed we did not want to store any cookies for legal issues.Hence we had to disable all the cookies.While testing the application on the test servers I noticed the following Asp.Net_sessionId cookie being set.

More about it here on msdn (http://msdn.microsoft.com/en-us/library/ms178194.ASPX)

We still didn't want to use it even though it was non persistent since we were not allowed to create any cookies.One workaround is to disable the session state on IIS for the site.

Note:You will not be able to use Session if you do this.

 

SqlDependency must be called for each server that is being executed

Recently we rewrote our web applications in VS 2012 and .Net 4.0 and hosted it on IIS 7.For caching content I used the Sql Dependency and SQL service Service Broker.On test environment with load balancing we had two servers configured and everything was working as expected.

However when I moved the applications to production I started getting this error.

The only difference was the connectionstring which specified a failover server.

<add name="CONTENTDSN" connectionString="Data Source=Server1;Failover Partner=Server2;Initial Catalog=Database_Name;uid=UserId;pwd=password;"/>

The error was due to the failover server not having the database objects.Now when you use SQL dependency make sure that the failover server i.e Server2, has all the necessary database objects as the main server i.e. Server1.

asp.net - Could not load type

Today while I was converting a Web Application to a WebSite in Visual Studio 2012 I came across this error.I created a new website and starting adding the pages from the web application.Once I was done adding the pages, I did a build on the website and I encountered the "Could not load type ' '" error in my website.

After searching for an answer I found this error can occur for various reasons, in my case though it was because of the CodeBehind attribute of the aspx pages.When you add a aspx page to a web application the CodeBehind attribute is set.

<%@ Page Title="About Page" Language="C#" MasterPageFile="~/Site.Master" AutoEventWireup="true" CodeBehind="About.aspx.cs" Inherits="About" %>

If you import this page into a website you will come across Could not load type About error.The fix was to change the attribute to CodeFile in the website aspx page

<%@ Page Title="About Page" Language="C#" MasterPageFile="~/Site.Master" AutoEventWireup="true" CodeFile="About.aspx.cs" Inherits="About" %>

See here for CodeFile vs CodeBehind
http://stackoverflow.com/questions/73022/codefile-vs-codebehind

c# - List strings in order of occurrence in a given string

In the project that I am working on right now I had to figure out a way to list a given set of strings in another string based on the order in which the list of strings would occur.

For example: I have a few raw html page content extracted from the web and I have a list of words.The task is to build a grid display that would list those words in order of occurrence in the content of a page. 

Below is the sample code using a console to achieve it.If there is a better way to do this, I am interested to know.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace StringOrder
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                //Set of random words to search for in Content
                string[] saArrayofStrings = { "abc", "def", "mno", "xyz", "jkl" };
                //Content
                string sString = "xyz The history of the United States as covered in American schools and universities typically begins with either Christopher Columbus's 1492 voyage to the Americas or with the prehistory of the Native peoples, with the latter approach having become increasingly common in recent decades.[1]jkl Indigenous peoples lived in what is now the United States for thousands of years and developed complex cultures before European colonists began to arrive, mostly from England, after 1600. The Spanish had early settlements in Florida and the Southwest, and the French along the Mississippi River and Gulf Coast.abc By the 1770s, thirteen British colonies contained two and a half million people along the Atlantic coast, east of the Appalachian Mountains.mno The colonies were prosperous and growing rapidly, and had developed their own autonomous political and legal systems. However, with the end of the French and Indian War in 1763, Great Britain altered its relationships with the colonies by imposing tighter administrative controls and greater financial obligations on the colonists [2]. Tensions grew, eventually leading to armed conflict beginning in April 1775. def On July 4, 1776, the colonies declared independence from the Kingdom of Great Britain.";
                SortedList<int, string> OrderList = new SortedList<int, string>();
                foreach (String s in saArrayofStrings)
                {
                    if (sString.Contains(s))
                        OrderList.Add(sString.IndexOf(s), s);
                }
                Console.WriteLine("The order of occurrence of the given list of words in the content is below");
                foreach (KeyValuePair<int, string> pair in OrderList)
                    Console.WriteLine(pair.Value + " occurs at the following position:" + pair.Key.ToString());
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            Console.Read();
        }
    }
}

Output: