Monday, October 30, 2006

Asynchronous LDAP Searching with System.DirectoryServices.Protocols

I have been hanging on to this post for some time now and never getting around to polishing it up and putting it out.  Eric Fleischman’s recent posting on SDS.P has inspired me to get some more content out there, however.  For whatever reason, this is a poorly documented topic that probably deserves a series of postings on it.  We neglected this topic mostly in the book because we had concerns about exceeding a certain # of pages on an already pretty dense topic.  I wonder if there would be any demand for a book on this subject?

Previously, I demonstrated how you need to perform paging in SDS.P using the cookie and the paging control.  There is a lot more to take care of here than how it is done with ADSI and System.DirectoryServices (SDS).  Now, what if we really wanted to perform this type of search asynchronously?  We actually wrote a sample in the book to show how asynchronous searching is done in SDS.P, but to keep it simple I left out the whole problem of paging.  Let’s be honest, any useful searching is probably going to need to handle paging as well.

SDS.P supports asynchronous searching directly, unlike SDS.  It is entirely possible to emulate this behavior in SDS by leveraging a general asynchronous pattern supported in .NET (e.g. by way of delegates, background thread workers, or the thread pool), but there is nothing in SDS to support asynchronously searching directly.  Hold on you say!  What about .NET 2.0 and the fancy new property on the DirectorySearcher called Asynchronous?  That is what we call a very misleading property.  While it actually sets the underlying ADS_SEARCHPREF_ASYNCHRONOUS option on the IDirectorySearch interface, the SDS model consumes the results synchronously, netting you exactly nothing.  I honestly wish they (the SDS .NET team) would just not have exposed that one if you couldn’t really use it.

Let’s look at the most basic pattern for how this can be accomplished:

  1. Create a connection to the LDAP directory (LdapConnection).
  2. Create a searching request operation (SearchRequest)
  3. Add a paging control to the SearchRequest to control paging
  4. Invoke the method asynchronously using LdapConnection.BeginSendRequest
  5. Provide a callback method to handle the results.

Simple, no?  It actually looks harder than it is.

When I set out creating this sample, I had a particular use in mind.  Specifically, I wanted to create an easy searching object – something where I wouldn’t have to worry about paging and that could also handle performing multiple searches on the same instance.  That last point, as you will see makes for additional complication.  I started with my usage model or how I visualized that I wanted to use it:

using (LdapConnection connection = CreateConnection(servername))
{
    AsyncSearcher searcher = CreateSearcher(connection);
 
    //this call is asynch, so we need to keep this main
    //thread alive in order to see anything
    //we can use the same searcher for multiple requests - we just have to track which one
    //is which, so we can interpret the results later in our events.
    _firstSearch = searcher.BeginPagedSearch(baseDN, "(sn=d*)", null, 500);
    _secondSearch = searcher.BeginPagedSearch(baseDN, "(sn=f*)", null, 500);
 
    //we will use a reset event to signal when we are done (using Sleep() on
    //current thread would work too...)
    _resetEvent.WaitOne(); //wait for signal;
}

I also knew that I would want to be notified of a couple key events that would occur with my object.  Specifically, whenever a page was returned and when the final search was complete:

static AsyncSearcher CreateSearcher(LdapConnection connection)
{
    AsyncSearcher searcher = new AsyncSearcher(connection);
 
    //assign some handlers for our events
    searcher.PageCompleted += new EventHandler<AsyncEventArgs>(searcher_PageCompleted);
    searcher.SearchCompleted += new EventHandler<AsyncEventArgs>(searcher_SearchCompleted);
 
    return searcher;
}

This would allow me to hook up a couple handlers that would be invoked each time a page came back or my search was finished.

static void searcher_SearchCompleted(object sender, AsyncEventArgs e)
{
    //this is volatile, so we need check it first or another thread
    //could change this from under us
    bool lastSearch = (((AsyncSearcher)sender).PendingSearches == 0);
 
    Console.WriteLine(
        "{0} Search Complete on thread {1}",
        e.RequestID.Equals(_firstSearch) ? "First" : "Second",
        Thread.CurrentThread.ManagedThreadId
        );
 
    if (lastSearch)
        _resetEvent.Set();
}
 
static void searcher_PageCompleted(object sender, AsyncEventArgs e)
{
    //or do something with e.Results here...
    Console.WriteLine(
        "Found {0} results on thread {1} for {2} search",
        e.Results.Count,
        Thread.CurrentThread.ManagedThreadId,
        e.RequestID.Equals(_firstSearch) ? "first" : "second"
        );
}

The complication with this model has to do with the fact that I wanted to be able to use the same object to search multiple times.  I could have limited my AsyncSearcher class to disallow more than one search at a time, but I thought that would be lame if I had to spin up a new instance and set of handlers for each search I wanted to perform.  Since I could only register for one paging and one completion event and yet multiple searches could be firing it, I needed a way to distinguish one search from another.  I decided to return a unique Guid for each search that could be matched up later to determine which search was activating the event.  With more time, I suppose I could have wrapped the Guid in something a little prettier or easier to use, but it suffices for this sample.

I also decided that I wanted to return results by pages as I got them as well as the huge block of them on completion of the search.  To do this, I created a simple class to hold the results called AsyncEventArgs:

/// <summary>
/// Just a simple class to hold some results
/// </summary>
public class AsyncEventArgs : EventArgs
{
    Guid _id;
    List<SearchResultEntry> _entries;
 
    public AsyncEventArgs(List<SearchResultEntry> entries, string requestID)
    {
        _entries = entries;
        _id = new Guid(requestID);
    }
 
    public List<SearchResultEntry> Results
    {
        get
        {
            return _entries;
        }
    }
 
    public Guid RequestID
    {
        get
        {
            return _id;
        }
    }
}

I could have created a different EventArg class depending on the type of event being fired, but I didn’t think it was worth it as this point.  Now that I had how I wanted to use the object nailed down a bit, I created the actual implementation.  The logic goes something like this in:

  1. Setup initial search and kick it off (steps 1–5 above), register a callback and pass my SearchRequest as state.
  2. Callback now picks up request and unwraps the SearchRequest (which also tells me which unique search I am after here) and calls LdapConnection.EndSendRequest()
  3. Pull the results from the resulting SearchResponse and fire the PageCompleted event – passing them as EventArgs
  4. Next, determine if the SearchResponse has a cookie and if it does, call BeginSendRequest again with updated paging cookie and point it back to my original callback (#2).  This means there is a lot of 2 through 4 going on here as each page gets processed and the PageCompleted event fires.
  5. If the cookie in #4 is empty, then I am done and I fire the SearchCompleted event and pass as EventArgs all the results I have been collecting internally in a hashtable keyed to the original request Guid.

Here is what that class looks like:

public class AsyncSearcher
{
    LdapConnection _connect;
    Hashtable _results = new Hashtable();
 
    public event EventHandler<AsyncEventArgs> SearchCompleted;
    public event EventHandler<AsyncEventArgs> PageCompleted;
 
    public AsyncSearcher(LdapConnection connection)
    {
        this._connect = connection;
        this._connect.AutoBind = true; //will bind on first search
    }
 
    /// <summary>
    /// Volatile count of outstanding searches in process
    /// </summary>
    public int PendingSearches
    {
        get
        {
            return _results.Count;
        }
    }
 
    private void InternalCallback(IAsyncResult result)
    {
        SearchResponse response = this._connect.EndSendRequest(result) as SearchResponse;
 
        ProcessResponse(response, ((SearchRequest)result.AsyncState).RequestId);
 
        //find the returned page response control
        foreach (DirectoryControl control in response.Controls)
        {
            if (control is PageResultResponseControl)
            {
                //call paged search again
                NextPage((SearchRequest)result.AsyncState, ((PageResultResponseControl)control).Cookie);
                break;
            }
        }
    }
 
    private void ProcessResponse(SearchResponse response, string guid)
    {
        //only 1 thread at a time gets here...
        List<SearchResultEntry> entries = new List<SearchResultEntry>();
 
        foreach (SearchResultEntry entry in response.Entries)
        {
            entries.Add(entry);
        }
 
        //signal our caller that we have a page
        EventHandler<AsyncEventArgs> OnPage = PageCompleted;
        if (OnPage != null)
        {
            OnPage(
                this,
                new AsyncEventArgs(entries, guid)
                );
        }
 
        //add to the main collection
        ((List<SearchResultEntry>)_results[guid]).AddRange(entries);
    }
 
    public Guid BeginPagedSearch(
        string baseDN,
        string filter,
        string[] attribs,
        int pageSize
        )
    {
        Guid guid = Guid.NewGuid();
 
        SearchRequest request = new SearchRequest(
            baseDN,
            filter,
            System.DirectoryServices.Protocols.SearchScope.Subtree,
            attribs
            );
 
        PageResultRequestControl prc = new PageResultRequestControl(pageSize);
 
        //add the paging control
        request.Controls.Add(prc);
 
        //we will use this to distinguish multiple searches.
        request.RequestId = guid.ToString();
 
        //create a temporary placeholder for the results
        _results.Add(request.RequestId, new List<SearchResultEntry>());
 
        //kick off async
        IAsyncResult result = this._connect.BeginSendRequest(
            request,
            PartialResultProcessing.NoPartialResultSupport,
            new AsyncCallback(InternalCallback),
            request
            );
 
        return guid;
    }
 
    private void NextPage(SearchRequest request, byte[] cookie)
    {
        //our last page is when the cookie is empty
        if (cookie != null && cookie.Length != 0)
        {
            //update the cookie and preserve page size
            foreach (DirectoryControl control in request.Controls)
            {
                if (control is PageResultRequestControl)
                {
                    ((PageResultRequestControl)control).Cookie = cookie;
                    break;
                }
            }
 
            //call it again to get next page
            IAsyncResult result = this._connect.BeginSendRequest(
                request,
                PartialResultProcessing.NoPartialResultSupport,
                new AsyncCallback(InternalCallback),
                request
                );
        }
        else
        {
            List<SearchResultEntry> results = (List<SearchResultEntry>)_results[request.RequestId];
 
            //decrement our collection when we are done
            _results.Remove(request.RequestId);
 
            //we have finished, signal the caller
            EventHandler<AsyncEventArgs> OnComplete = SearchCompleted;
            if (OnComplete != null)
            {
                OnComplete(
                    this,
                    new AsyncEventArgs(results, request.RequestId)
                    );
            }
        }
    }
}

I originally toyed with the idea of having the AsyncSearcher manage the connection to the directory, but ultimately decided that it was a bad idea for two main reasons.  First, not everyone will construct the connection the same.  Some people will use SSPI, others SSL, perhaps different ports, even certificates or otherwise.  The other reason is that the lifetime of the connection is harder to control from inside another class.  Wrapping it in a “using” statement won’t work because the connection must be open the whole time during the callbacks.  Cleaning it up in a Dispose() method is sloppy as well because it leads to the client needing to know that they should keep this object around until all their events fire.  By pulling the LdapConnection out, I am implicitly telling clients they need to manage the connection themselves.

You can download this class along with a sample client as an attachment to this post.  Next time, perhaps we will delve into retrieving partial results.

 File Attachment: AsynchClient.zip (5 KB).

Comments are closed.