Thanksgiving, 2008

As we come to Thanksgiving, here are some things I'm thankful for this year.

• My wife, children, and I had another year of good health.
  
• The country managed to get through the election without another legal fiasco.
• I found a new job (rather, a new job found me) relatively quickly after being laid off.
• We made the decision to not buy a new car or commit to major home renovations mere months before being laid off and this current economic zaniness.
• My sister made it through her 3rd tour of duty in Iraq, her 4th overseas, in one piece.

Happy Thanksgiving!

Loading XML into eXist Using XQuery and the Sandbox

This past weekend I was tinkering with the eXist XML database. The installation went fine and some of their sample queries ran fine. My next step was to load some of my content into it.

Rather than use their web interface or desktop client, I wanted to load the documents using XQuery through their sandbox application. I thought this would be quick and easy and would allow me to compare some features of eXist to MarkLogic Server.

There is quite a bit of documentation for eXist, but the XQuery API is light on specific usage examples. I also ran into some non-obvious gotchas. Here is the XQuery code that I used to load a document into a specific collection, along with some notes below.

declare namespace xmldb="http://exist-db.org/xquery/xmldb";
declare variable $file as xs:string { 
  "file:///C:/Program%20Files/eXist/samples/mattio/sample.xml" };
declare variable $name as xs:string { "sample.xml" };
declare variable $collection as xs:string { "/db/test/" };

<results>
{
let $collection-status := 
  if(not(xmldb:collection-exists($collection))) then 
    xmldb:create-collection("", $collection)
  else ("Collection already exists.")
return <collection-status> { $collection-status } </collection-status>
,
let $load-status := xmldb:store($collection, $name, xs:anyURI($file))
return <load-status> { $load-status } </load-status>
}
</results>

When I was trying to use C:\ to start my path or when I was leaving out xs:anyURI(), I was getting a misleading error that implied there was something wrong with my document. The error was:

XMLDB reported an exception while storing documentorg.xmldb.api.base.XMLDBException: fatal error at (1,1) : Content is not allowed in prolog. [at line 120, column 21] In call to function: sandbox:exec-query(xs:string) [134:10]

Here are some other notes.

1. Note that the xmldb namespace needs to be declared.


2. Note the syntax of $file. This is how you reference a document on your file system, including encoding the path to use %20 instead of a space.


3. Note that $file must be wrapped in xs:anyURI() when used in xmldb:store() in order to force it to be considered a URI and not a simple string.

Thanks to Dannes and Wolfgang for their help with this. They were on the exist-open list on a Saturday.

Next up I'll load about 50 large documents to build some basic queries to review index tuning.

Exporting XML Files from Textml

I had a case where documents were being created and stored dynamically in Textml Server by an application, but we wanted the physical files exported. I had a ContentServer class already in place for selecting all documents in a collection and for selecting a document by file name, which would make this easier. This was going nowhere near a production server, so reusing what I had to get this done quickly was my primary concern.

There are some ways to clean this up, but the general approach should be helpful in similar situations.

Here is the method that will return a list of all documents in a collection. This gives me the file name, which I use to get the individual documents.

private List<ListItem> SelectAllTextml()
{
    List<ListItem> myList = new List<ListItem>();
    string textmlStandardHeader = "<?xml version=\"1.0\" encoding=\"UTF-16\"?><query VERSION=\"3.6\" RESULTSPACE=\"RGuideAdmin\">\n";
    string textmlStandardFooter = "</query>";
    string textmlCollection = "<property NAME=\"collection\"><elem>" +
      this.ContextAdditionalName + "</elem></property>";
    string textmlFile = "<property NAME=\"NAME\"><elem><anystr/></elem></property>";
    string textmlQuery = textmlStandardHeader + "<andkey>" +
        textmlCollection +
        textmlFile +
        "</andkey>" + textmlStandardFooter;
    IxiaClientServices IxiaCS = new IxiaClientServices();
    IxiaServerServices IxiaSS = IxiaCS.ConnectServer(this.ContextServer);
    IxiaDocBaseServices IxiaDS = IxiaSS.ConnectDocBase(this.ContextContainer);
    IxiaSearchServices IxiaSearchS = IxiaDS.SearchServices;
    IxiaResultSpace textmlResultSpace = IxiaSearchS.SearchDocuments(textmlQuery);
    if (textmlResultSpace.Count > 0)
    {
        for (int i = 0; i < textmlResultSpace.Count; i++)
        {
            ListItem documentItem = new ListItem();                
            IxiaDocument document;
            document = textmlResultSpace.Item(i);

            MemoryStream xmlStream = new MemoryStream();
            document.Content.SaveTo(xmlStream);
            xmlStream.Position = 0;
            XPathDocument textmlXmlDocument = new XPathDocument(xmlStream);
            XPathNavigator textmlXmlNav = textmlXmlDocument.CreateNavigator();
            documentItem.Text = 
              textmlXmlNav.SelectSingleNode("descendant::title[1]").ToString() + 
              " (" + document.Collection + ")";
            documentItem.Value = 
              textmlXmlNav.SelectSingleNode("descendant::guide[1]").
              GetAttribute("id", "");

            myList.Add(documentItem);
        }
    }
    return myList;
}

Here is the method that will return a document (or documents) by file name, limited to a collection.

private List<XmlDocument> SelectTextml(string fileName)
{
    List<XmlDocument> myList = new List<XmlDocument>();
    string textmlStandardHeader = "<?xml version=\"1.0\" encoding=\"UTF-16\"?><query VERSION=\"3.6\" RESULTSPACE=\"RGuideAdmin\">\n";
    string textmlStandardFooter = "</query>";
    string textmlCollection = "<property NAME=\"collection\"><elem>" +
      this.ContextAdditionalName + "</elem></property>";
    string textmlFile = "<property NAME=\"NAME\"><elem>" + fileName + "<anystr/></elem></property>";
    string textmlQuery = textmlStandardHeader + "<andkey>" +
        textmlCollection +
        textmlFile +
        "</andkey>" + textmlStandardFooter;
    IxiaClientServices IxiaCS = new IxiaClientServices();
    IxiaServerServices IxiaSS = IxiaCS.ConnectServer(this.ContextServer);
    IxiaDocBaseServices IxiaDS = IxiaSS.ConnectDocBase(this.ContextContainer);
    IxiaSearchServices IxiaSearchS = IxiaDS.SearchServices;
    IxiaResultSpace textmlResultSpace = IxiaSearchS.SearchDocuments(textmlQuery);
    if (textmlResultSpace.Count > 0)
    {
        for (int i = 0; i < textmlResultSpace.Count; i++)
        {
            IxiaDocument document = textmlResultSpace.Item(i);
            MemoryStream xmlStream = new MemoryStream();
            document.Content.SaveTo(xmlStream);
            xmlStream.Position = 0;
            XmlDocument textmlXmlDocument = new XmlDocument();
            textmlXmlDocument.Load(xmlStream);
            myList.Add(textmlXmlDocument);
        }
    }

    return myList;
}

Here's the method I used to go through each document returned in the collection list and save each to an XML file.

myContentServer = new ContentServer(Server.MapPath("~/App_Data/" + 
  ddlProduct.SelectedValue + ".xml"));
List<ListItem> myGuides = myContentServer.SelectAll();
if (myGuides.Count > 0)
{
    if (Directory.Exists(Server.MapPath(exportDirectory + "/" + 
        ddlProduct.SelectedValue)))
    {
        // Delete the directory and anything existing in it.
        Directory.Delete(Server.MapPath(exportDirectory + "/" + 
          ddlProduct.SelectedValue), true);         
    }
    Directory.CreateDirectory(Server.MapPath(exportDirectory + "/" + 
      ddlProduct.SelectedValue));
    foreach (ListItem guide in myGuides)
    {
        List<XmlDocument> myGuide = myContentServer.Select(guide.Value);
        XmlDocument document = myGuide[0];
        // Save the file to the export directory. 
        document.Save(Server.MapPath(exportDirectory + "/" + 
          ddlProduct.SelectedValue + "/" + guide.Value + ".xml"));
        divExportList.InnerHtml += "<br/>" + exportDirectory + "/" + 
          ddlProduct.SelectedValue + "/" + guide.Value + ".xml";
    }
}

Weird Bug While Porting Textml Server Code from JSP to ASP.NET

This morning I was porting an old search results page accessing Textml Server from JSP to ASP.NET. One feature implemented there is search within results. We execute this by storing the original query in the session and then, when a user asks to search within results, we pull it out and re-run it so the second query can reference the first.

We have a line like this in the JSP page...

IxiaResultSpace originalResults = 
  search.SearchDocuments((String)session.getAttribute("resultQuery"));

...followed by a few lines later by a line like this...

"<include TYPE=\"ResultSpace\">" + sessionID + "-ALL</include>"

All was well.

The logic of the page overall is more than a bit wonky, but we decided to port first and revise later. When done, I was getting an error that said

"vrn2nc55cxej5knnemwyzvqv-ALL is not a valid ResultSpace include /query/andkey/include at Ixiasoft.TextmlServer.ResultSpace.ExecuteQuery() at Ixiasoft.TextmlServer.ResultSpace.get_Count() at searchresults.RunSearch() in c:\Greenwood Web Sites\devsite\searchresults.aspx.cs:line 357."

What's that now?

After some trips through the debugger, poking around the documentation and some googling (no one blogs on this thing) I went back to the old method of just writing out strings to the page. Nothing jumped out as an error and nothing worked.

By sheer chance, I decided to see what the string value of the original query was so I added...

string originalQuery = originalResults.TextmlQuery;

...to the page with the intent of displaying it somewhere for review and suddenly the error stopped being thrown and the code functioned as expected. After making sure I made no other changes I tested it again. I commented out that line and the error was thrown. I put the line back in and the page ran fine. A co-worker asked if the Count property forced it work as well and it does.

I can't explain this one.

Logins Fail after SQL Server Restore

After doing a SQL Server db restore, logins can be a problem. This script will re-sync the passwords.

EXEC sp_change_users_login 'Auto_Fix','UserOne', null, 'pwd1'
EXEC sp_change_users_login 'Auto_Fix','UserTwo', null, 'pwd2'
EXEC sp_change_users_login 'Auto_Fix','UserThree', null, 'pwd3'

To find these users.

EXEC sp_change_users_login 'Report', null, null, null