Annotation Import web service error

During a recent import I was getting some exceptions while adding annotations to Accounts and Contacts. The exceptions being generated were not SoapExceptions, but HTTP 400 malformed http request errors.

This was a data driven problem and I assumed that it was an illegal character messing up the SOAP. The annotation.notetext attribute was the only one being set to a large chunk of unknown text. Debugging this showed some illegal characters in my source data. Initially I found an occasional "\0" which is easily removed, but there was another character that showed up as a box symbol.

So how do you filter that out?

First you need to know what you are looking at, so while debugging copy the offending character into the clipboard and paste it into a text file. In Visual Studio there is a binary editor.

File -> Open File
After selecting your file click notice the small pick list to the right of Open.
Select Open With...

Viewing the text file revealed that the offending character was a hex 12. I had copied three of them into NotePad. For the curious among you a hex 12 = DC2 or Device Control 2.

Armed with that information the following statement filtered out the offending characters and the import completed without error.

newAnnotation.notetext = note.Replace('\0', ' ').Replace('\u0012', ' ');

Below is a code example using this filter to generate contact annotations.

var newAnnotation = new annotation
       {
          objectid = CrmTypes.CreateLookup(EntityName.contact.ToString(), contactId),
          objecttypecode = CrmTypes.CreateEntityNameReference(EntityName.contact.ToString()),
          isdocument = CrmTypes.CreateCrmBoolean(false),
          ownerid = newContact.ownerid,
          subject = "GoldMine notes",
          notetext = note.Replace('\0', ' ').Replace('\u0012', ' ')
       };

try
{
    service.Create(newAnnotation);
}

This could have been solved a number of other ways like a regex that only allows valid characters for a SOAP request, but I haven't run into this issue enough to create one.