The DataImport Example

The DataImport Example

At this point we are ready to concentrate our attention on a practical example that will be useful to demonstrate what we have seen thus far. The DataImport example, included here, is a typical application that waits for files to arrive in a specific directory before importing them into a SQL Server database. The code for this application, as with the rest of the code in this book, can be found at the Apress web site. Below we outline the classes that will be used in this example:

FileSystemWatcher: This allows developers to specify the directory to monitor and to raise an event when something changes (for example, a new file is created or removed). This class is contained in the System.IO namespace of the .NET Framework class library.

TextWriterTraceListener: This implements our own tracing functionality.

Thread: This, which you've seen many times before, allows us to start a new thread to import data into the database.

Many classes from the SqlClient namespace necessary to manage the SQL Server database connection and update.

The first release of the DataImport application contains some logical errors that you will discover using tracing functionality. In that way you can have a good example about log (trace) files and their importance.

To learn more about the ADO.NET classes, please refer to Professional ADO.NET Programming (ISBN 1-86100-527-X), or ADO.NET Programmer's Reference (ISBN 1-86100-558-X).

The Code

Let's start analyzing the code of the DataImport example:

using System;
using System.IO;
using System.Data;
using System.Data.SqlClient;
using System.Threading;
using System.Diagnostics;

namespace DataImport1

The above code implements an infinite loop, which waits for the file creation event to be raised. The WaitForChangedResult object will contain information about the file created. For example, the code uses the Name property to trace the name of the discovered file.

catch (Exception e)

finally

The above Main method ends by tracing some useful messages and any exceptions. The OnFileCreated() static method is detailed below:

private static void OnFileCreated(Object source,
FileSystemEventArgs e)

finally


class ImportData

catch (Exception ex)

finally

Then, after writing the code for catching and dealing with any exceptions that may occur, the code is complete.

Testing the Application

To test the application you have to follow these steps:

Create a C:temp directory to contain the XML file

Run the DataImport application

Copy the authors.xml file into the C:temp directory

As a final result you should find the DataImport.log file in the C:directory having content similar to this:

01/05/2002 12:23:01 - Found: authors.xml file
01/05/2002 12:23:01 - Filling the DataSet.
01/05/2002 12:23:02 - Reading XML file.
01/05/2002 12:23:02 - DataSet filled with data.
01/05/2002 12:23:02 - Updating database.
01/05/2002 12:23:02 - Database updated successfully.
01/05/2002 12:23:03 - Total TIME: 0 second/s

The authors.xml file is not that large so the total time is less than one second.

Logical Errors

All seems to be working well, but obviously, everything hasn't been accounted for. So far, we have tested our application with a very small file size, so when the application receives the file creation event and opens the file, the process that copies it into the directory finishes its task of closing the file. What happens when you receive a huge file? Well, when the thread tries to access the XML file and fill the DataSet object, it receives an access-denied error caused by attempting to open a file already in use by the copier task. Try to test the application again by copying the huge_authors.xml file instead. Since you have used tracing messages, you may find the following error in the log file:

4/14/2002 1:29:00 PM - Found: huge_authors.xml file
4/14/2002 1:29:00 PM - Filling the DataSet.
4/14/2002 1:29:00 PM - Reading XML file.
4/14/2002 1:29:00 PM - A general exception occurred during file processing:
4/14/2002 1:29:00 PM - System.IO.IOException: The process cannot access the file
'C:temphuge_authors.xml' because it is being used by another process.
at System.IO.__Error.WnIOError(Int32 errorCode, String str)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare
share, Int32 bufferSize, Boolean useAsync, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare
share)
at System.Xml.XmlDownloadManager.GetStream(Uri uri, (Credentials credentials)
at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn)
at System.Xml.XmlTextReader.CreateScanner()
at System.Xml.XmlTextReader.Init()
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlReader.MoveToContent()
at System.Data.DataSet.ReadXml(XmlReader reader, XmlReadMode mode)
at System.Data.DataSet.ReadXml(String fileName, XmlReadMode mode)

This is a kind of error that the debugger often fails to catch because the time used to launch it and the time to step through the code is often sufficient to copy the file. It may also not occur on your machine. It depends on the speed of your disk access and the amount of memory you have (so how much the application is slowed down).

The error message suggests a possible solution that you should add to the application to resolve the error. Before calling the ReadXml() method, you should try to open the file with exclusive access. If an error occurs, then you can suspend the thread for few seconds, trying again when the file can be processed. Let's see how the code changes in DataImport2, by adding the GetFileAccess() method:

private bool GetFileAccess()

catch

The GetFileAccess() method has been added in order to return a Boolean value indicating whether you can have exclusive access to the file or not. The method simply tries to open the file with the share access property set to None:

public void Import()


Trace.WriteLineIf(DataImport.bs.Enabled, DateTime.Now +
' - Filling the DataSet.');
// Fill a dataset with data within the
// authors table
da.Fill(ds);

The Import() method provided by the ImportData class will try to get exclusive access to the file. If the file is still opened by the copier task, the thread will be suspended for five seconds. So, the GetFileAccess() method will be called until the source file can be opened.

We have seen practically how the tracing functionalities can be useful to understand the application behavior during run-time execution.