In one of the projects I am converting to .Net 4.0, I started getting the following error 'The process cannot access the file because it is being used by another process'. The program is processing a bunch of files in a set of folders. I am using .Net 4.0 Task Parallel Library features to handle these folders across different threads on the server. The server has 16 GB RAM and 24 cores.
The code that was causing the error is below
Parallel.ForEach(rangePartitioner,new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount}, (range,loopState) =>
{
try
{
Parallel.ForEach(rangePartitioner,new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount}, (range,loopState) =>
{
try
{
using (FileStream oFileStream = new FileStream(oFileInfo.FullName, FileMode.Open,FileAccess.Read, FileShare.Read)))
using (GZipStream oGzipStream = new GZipStream(oFileStream, CompressionMode.Decompress))
using (StreamReader oStreamReader = new StreamReader(oGzipStream, Encoding.ASCII))
{
String sRawDataLine = oStreamReader.ReadLine();
while (sRawDataLine != null)
{
//Do Something();
sRawDataLine = oStreamReader.ReadLine();
}
}
}
catch (Exception ex)
{
oLog.WriteLine("Error processing file name = {0} Exception {1}.", oFileInfo.FullName, ex.ToString());
}
});
I have not seen the error since adding these two options.The process no longer fails and executes on all the 24 cores on the server without any issues.
The code that was causing the error is below
Parallel.ForEach(rangePartitioner,new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount}, (range,loopState) =>
{
try
{
using (FileStream oFileStream = new FileStream(oFileInfo.FullName, FileMode.Open))
using (GZipStream oGzipStream = new GZipStream(oFileStream, CompressionMode.Decompress))
using (StreamReader oStreamReader = new StreamReader(oGzipStream, Encoding.ASCII))
{
String sRawDataLine = oStreamReader.ReadLine();
while (sRawDataLine != null)
{
//Do Something();
sRawDataLine = oStreamReader.ReadLine();
}
}
}
catch (Exception ex)
{
oLog.WriteLine("Error processing file name = {0} Exception {1}.", oFileInfo.FullName, ex.ToString());
}
});
The "key" for the "locks" was to add the following options highlighted in the below code to the FileStream object so the file access is for read operation and that the file could be shared while reading.
{
try
{
using (FileStream oFileStream = new FileStream(oFileInfo.FullName, FileMode.Open,FileAccess.Read, FileShare.Read)))
using (GZipStream oGzipStream = new GZipStream(oFileStream, CompressionMode.Decompress))
using (StreamReader oStreamReader = new StreamReader(oGzipStream, Encoding.ASCII))
{
String sRawDataLine = oStreamReader.ReadLine();
while (sRawDataLine != null)
{
//Do Something();
sRawDataLine = oStreamReader.ReadLine();
}
}
}
catch (Exception ex)
{
oLog.WriteLine("Error processing file name = {0} Exception {1}.", oFileInfo.FullName, ex.ToString());
}
});
I have not seen the error since adding these two options.The process no longer fails and executes on all the 24 cores on the server without any issues.
No comments:
Post a Comment