Friday, October 17, 2008

Creating Attachments to emails and annotations

MS CRM stores attachments directly in the database which allows security to be enforced on attachments. If you delete a entity, all of its annotations are removed with it. By way of comparison this offers a lot more control than just saving a pathname for an attached file like GoldMine does with its RFC822 format emails.

In MS CRM 4.0 the attachments are stored in a text field in the ActivityMimeAttachment table, and in an nvarchar(max) in the AnnotationBase table.  You can use SQL "like" comparator in a where clause against an nvarchar but not for a text field. nvarchar(max) was introduced with SQL 2005.

Creating an activitymimeattachment ( Email attachment )

The following is a code snippet from a larger method that creates the email entity as well. This code is inside a try catch for the email creation.

Notice that a file can be converted into a format that can be saved into the activitymimeattachment.body attribute by using the File.ReadAllBytes() and the Convert.ToBase64String() methods.

// If you have an email entity ready to be created and list of pathnames
// the following code will generate the related activitymimeattachment entities 

Guid emailId = service.Create(newEmail);
 
 if (attachmentPathnames.Count > 0)
 {
     int attachmentCount = 1;

     foreach (var attachmentPathName in attachmentPathnames)
    {
     var emailAttachment = new activitymimeattachment
             {
                // Relate back to email activity 
                activityid = CrmTypes.CreateLookup(EntityName.email.ToString(), emailId),
                filename = attachmentPathName.Substring(attachmentPathName.LastIndexOf('\\') + 1), 
                subject = subject,
                attachmentnumber = CrmTypes.CreateCrmNumber(attachmentCount ++)
             };

     if (File.Exists(attachmentPathName))
     {         
         byte[] fileContents = File.ReadAllBytes(attachmentPathName);
         emailAttachment.body = Convert.ToBase64String(fileContents);
         emailAttachment.filesize = CrmTypes.CreateCrmNumber((int) new FileInfo(attachmentPathName).Length);
         emailAttachment.mimetype = GetMimeType(attachmentPathName); // GetMimeType() Covered Below
     }
     else
     {
         // Handle condition where file doesn't exist
         emailAttachmentbody = "Attachment File Not Found.";
     }

     try
     {
         service.Create(emailAttachment);
     }

     catch (SoapException ex)
     {
         string errormsg = String.Format("Write Error for Email Attachment {0} error {1}", attachmentPathName, ex.Detail.InnerText);
         WriteErrorFile(errormsg);
     }
    }
 }

Creating an annotation (Notes attachment)

The following code creates an annotation which can belong to any entity with a Notes tab. In this case it is a Letter activity. This is very similar to the above code for the activitymimeattachment. One key difference is that an annotation can also just be a Note,  so it has an attribute just to specify if it is a note or an attachment.

Notice the isdocument attribute of the annotation. This needs to be set to true for an attachment. Also notice the overriddencreatedon attribute. The addition of this is one of the many improvements in MS CRM 4.0 and removed one of my pet peeves about MS CRM 3.0. When importing annotations in MS CRM 3.0 all of the annotations would be set to the import date which is useless for the end user. The ONLY way to change it was to go back with a SQL Update to change the createdon date. Now we have the overriddencreatedon attribute so that we don't have to "break the rules" and access the data tables directly.

Guid letterId = service.Create(newLetter); var newAnnotation = new annotation { objectid = CrmTypes.CreateLookup(EntityName.letter.ToString(), letterId), objecttypecode = CrmTypes.CreateEntityNameReference(EntityName.letter.ToString()), isdocument = CrmTypes.CreateCrmBoolean(false), ownerid = newLetter.ownerid, subject = subject, overriddencreatedon = newLetter.actualend, notetext = "File from import." }; if (hasAttachment) { if (File.Exists(attachmentPath)) { newAnnotation.isdocument = CrmTypes.CreateCrmBoolean(true), newAnnotation.filename = attachmentPath.Substring(attachmentPath.LastIndexOf('\\') + 1); byte[] fileContents = File.ReadAllBytes(attachmentPath); newAnnotation.documentbody = System.Convert.ToBase64String(fileContents); newAnnotation.filesize = CrmTypes.CreateCrmNumber((int)new FileInfo(attachmentPath).Length); newAnnotation.mimetype = GetMimeType(attachmentPath); } else { // Handle missing attachment
newAnnotation.subject += " - (attached document not found)"; } try { service.Create(newAnnotation); } catch (SoapException ex) { string errormsg = String.Format("Write Error for Letter Attachment {0} error {1}", attachmentPath, ex.Detail.InnerText); WriteErrorFile(errormsg); } }

The following method is a helper to look up a mimetype for a given file extension.

public string GetMimeType(string fileName)
{
    string mimeType;

    switch (System.IO.Path.GetExtension(fileName).ToLower())
    {
        case ".3dm": mimeType = "x-world/x-3dmf"; break;
        case ".3dmf": mimeType = "x-world/x-3dmf"; break;
        case ".a": mimeType = "application/octet-stream"; break;
        case ".aab": mimeType = "application/x-authorware-bin"; break;
        case ".aam": mimeType = "application/x-authorware-map"; break;
        case ".aas": mimeType = "application/x-authorware-seg"; break;
        case ".abc": mimeType = "text/vnd.abc"; break;
        case ".acgi": mimeType = "text/html"; break;
        case ".afl": mimeType = "video/animaflex"; break;
        case ".ai": mimeType = "application/postscript"; break;
        case ".aif": mimeType = "audio/aiff"; break;
        case ".aifc": mimeType = "audio/aiff"; break;
        case ".aiff": mimeType = "audio/aiff"; break;
        case ".aim": mimeType = "application/x-aim"; break;
        case ".aip": mimeType = "text/x-audiosoft-intra"; break;
        case ".ani": mimeType = "application/x-navi-animation"; break;
        case ".aos": mimeType = "application/x-nokia-9000-communicator-add-on-software"; break;
        case ".aps": mimeType = "application/mime"; break;
        case ".arc": mimeType = "application/octet-stream"; break;
        case ".arj": mimeType = "application/arj"; break;
        case ".art": mimeType = "image/x-jg"; break;
        case ".asf": mimeType = "video/x-ms-asf"; break;
        case ".asm": mimeType = "text/x-asm"; break;
        case ".asp": mimeType = "text/asp"; break;
        case ".asx": mimeType = "video/x-ms-asf"; break;
        case ".au": mimeType = "audio/basic"; break;
        case ".avi": mimeType = "video/avi"; break;
        case ".avs": mimeType = "video/avs-video"; break;
        case ".bcpio": mimeType = "application/x-bcpio"; break;
        case ".bin": mimeType = "application/octet-stream"; break;
        case ".bm": mimeType = "image/bmp"; break;
        case ".bmp": mimeType = "image/bmp"; break;
        case ".boo": mimeType = "application/book"; break;
        case ".book": mimeType = "application/book"; break;
        case ".boz": mimeType = "application/x-bzip2"; break;
        case ".bsh": mimeType = "application/x-bsh"; break;
        case ".bz": mimeType = "application/x-bzip"; break;
        case ".bz2": mimeType = "application/x-bzip2"; break;
        case ".c": mimeType = "text/plain"; break;
        case ".c++": mimeType = "text/plain"; break;
        case ".cat": mimeType = "application/vnd.ms-pki.seccat"; break;
        case ".cc": mimeType = "text/plain"; break;
        case ".ccad": mimeType = "application/clariscad"; break;
        case ".cco": mimeType = "application/x-cocoa"; break;
        case ".cdf": mimeType = "application/cdf"; break;
        case ".cer": mimeType = "application/pkix-cert"; break;
        case ".cha": mimeType = "application/x-chat"; break;
        case ".chat": mimeType = "application/x-chat"; break;
        case ".class": mimeType = "application/java"; break;
        case ".com": mimeType = "application/octet-stream"; break;
        case ".conf": mimeType = "text/plain"; break;
        case ".cpio": mimeType = "application/x-cpio"; break;
        case ".cpp": mimeType = "text/x-c"; break;
        case ".cpt": mimeType = "application/x-cpt"; break;
        case ".crl": mimeType = "application/pkcs-crl"; break;
        case ".crt": mimeType = "application/pkix-cert"; break;
        case ".csh": mimeType = "application/x-csh"; break;
        case ".css": mimeType = "text/css"; break;
        case ".cxx": mimeType = "text/plain"; break;
        case ".dcr": mimeType = "application/x-director"; break;
        case ".deepv": mimeType = "application/x-deepv"; break;
        case ".def": mimeType = "text/plain"; break;
        case ".der": mimeType = "application/x-x509-ca-cert"; break;
        case ".dif": mimeType = "video/x-dv"; break;
        case ".dir": mimeType = "application/x-director"; break;
        case ".dl": mimeType = "video/dl"; break;
        case ".doc": mimeType = "application/msword"; break;
        case ".dot": mimeType = "application/msword"; break;
        case ".dp": mimeType = "application/commonground"; break;
        case ".drw": mimeType = "application/drafting"; break;
        case ".dump": mimeType = "application/octet-stream"; break;
        case ".dv": mimeType = "video/x-dv"; break;
        case ".dvi": mimeType = "application/x-dvi"; break;
        case ".dwf": mimeType = "model/vnd.dwf"; break;
        case ".dwg": mimeType = "image/vnd.dwg"; break;
        case ".dxf": mimeType = "image/vnd.dwg"; break;
        case ".dxr": mimeType = "application/x-director"; break;
        case ".el": mimeType = "text/x-script.elisp"; break;
        case ".elc": mimeType = "application/x-elc"; break;
        case ".env": mimeType = "application/x-envoy"; break;
        case ".eps": mimeType = "application/postscript"; break;
        case ".es": mimeType = "application/x-esrehber"; break;
        case ".etx": mimeType = "text/x-setext"; break;
        case ".evy": mimeType = "application/envoy"; break;
        case ".exe": mimeType = "application/octet-stream"; break;
        case ".f": mimeType = "text/plain"; break;
        case ".f77": mimeType = "text/x-fortran"; break;
        case ".f90": mimeType = "text/plain"; break;
        case ".fdf": mimeType = "application/vnd.fdf"; break;
        case ".fif": mimeType = "image/fif"; break;
        case ".fli": mimeType = "video/fli"; break;
        case ".flo": mimeType = "image/florian"; break;
        case ".flx": mimeType = "text/vnd.fmi.flexstor"; break;
        case ".fmf": mimeType = "video/x-atomic3d-feature"; break;
        case ".for": mimeType = "text/x-fortran"; break;
        case ".fpx": mimeType = "image/vnd.fpx"; break;
        case ".frl": mimeType = "application/freeloader"; break;
        case ".funk": mimeType = "audio/make"; break;
        case ".g": mimeType = "text/plain"; break;
        case ".g3": mimeType = "image/g3fax"; break;
        case ".gif": mimeType = "image/gif"; break;
        case ".gl": mimeType = "video/gl"; break;
        case ".gsd": mimeType = "audio/x-gsm"; break;
        case ".gsm": mimeType = "audio/x-gsm"; break;
        case ".gsp": mimeType = "application/x-gsp"; break;
        case ".gss": mimeType = "application/x-gss"; break;
        case ".gtar": mimeType = "application/x-gtar"; break;
        case ".gz": mimeType = "application/x-gzip"; break;
        case ".gzip": mimeType = "application/x-gzip"; break;
        case ".h": mimeType = "text/plain"; break;
        case ".hdf": mimeType = "application/x-hdf"; break;
        case ".help": mimeType = "application/x-helpfile"; break;
        case ".hgl": mimeType = "application/vnd.hp-hpgl"; break;
        case ".hh": mimeType = "text/plain"; break;
        case ".hlb": mimeType = "text/x-script"; break;
        case ".hlp": mimeType = "application/hlp"; break;
        case ".hpg": mimeType = "application/vnd.hp-hpgl"; break;
        case ".hpgl": mimeType = "application/vnd.hp-hpgl"; break;
        case ".hqx": mimeType = "application/binhex"; break;
        case ".hta": mimeType = "application/hta"; break;
        case ".htc": mimeType = "text/x-component"; break;
        case ".htm": mimeType = "text/html"; break;
        case ".html": mimeType = "text/html"; break;
        case ".htmls": mimeType = "text/html"; break;
        case ".htt": mimeType = "text/webviewhtml"; break;
        case ".htx": mimeType = "text/html"; break;
        case ".ice": mimeType = "x-conference/x-cooltalk"; break;
        case ".ico": mimeType = "image/x-icon"; break;
        case ".idc": mimeType = "text/plain"; break;
        case ".ief": mimeType = "image/ief"; break;
        case ".iefs": mimeType = "image/ief"; break;
        case ".iges": mimeType = "application/iges"; break;
        case ".igs": mimeType = "application/iges"; break;
        case ".ima": mimeType = "application/x-ima"; break;
        case ".imap": mimeType = "application/x-httpd-imap"; break;
        case ".inf": mimeType = "application/inf"; break;
        case ".ins": mimeType = "application/x-internett-signup"; break;
        case ".ip": mimeType = "application/x-ip2"; break;
        case ".isu": mimeType = "video/x-isvideo"; break;
        case ".it": mimeType = "audio/it"; break;
        case ".iv": mimeType = "application/x-inventor"; break;
        case ".ivr": mimeType = "i-world/i-vrml"; break;
        case ".ivy": mimeType = "application/x-livescreen"; break;
        case ".jam": mimeType = "audio/x-jam"; break;
        case ".jav": mimeType = "text/plain"; break;
        case ".java": mimeType = "text/plain"; break;
        case ".jcm": mimeType = "application/x-java-commerce"; break;
        case ".jfif": mimeType = "image/jpeg"; break;
        case ".jfif-tbnl": mimeType = "image/jpeg"; break;
        case ".jpe": mimeType = "image/jpeg"; break;
        case ".jpeg": mimeType = "image/jpeg"; break;
        case ".jpg": mimeType = "image/jpeg"; break;
        case ".jps": mimeType = "image/x-jps"; break;
        case ".js": mimeType = "application/x-javascript"; break;
        case ".jut": mimeType = "image/jutvision"; break;
        case ".kar": mimeType = "audio/midi"; break;
        case ".ksh": mimeType = "application/x-ksh"; break;
        case ".la": mimeType = "audio/nspaudio"; break;
        case ".lam": mimeType = "audio/x-liveaudio"; break;
        case ".latex": mimeType = "application/x-latex"; break;
        case ".lha": mimeType = "application/octet-stream"; break;
        case ".lhx": mimeType = "application/octet-stream"; break;
        case ".list": mimeType = "text/plain"; break;
        case ".lma": mimeType = "audio/nspaudio"; break;
        case ".log": mimeType = "text/plain"; break;
        case ".lsp": mimeType = "application/x-lisp"; break;
        case ".lst": mimeType = "text/plain"; break;
        case ".lsx": mimeType = "text/x-la-asf"; break;
        case ".ltx": mimeType = "application/x-latex"; break;
        case ".lzh": mimeType = "application/octet-stream"; break;
        case ".lzx": mimeType = "application/octet-stream"; break;
        case ".m": mimeType = "text/plain"; break;
        case ".m1v": mimeType = "video/mpeg"; break;
        case ".m2a": mimeType = "audio/mpeg"; break;
        case ".m2v": mimeType = "video/mpeg"; break;
        case ".m3u": mimeType = "audio/x-mpequrl"; break;
        case ".man": mimeType = "application/x-troff-man"; break;
        case ".map": mimeType = "application/x-navimap"; break;
        case ".mar": mimeType = "text/plain"; break;
        case ".mbd": mimeType = "application/mbedlet"; break;
        case ".mc$": mimeType = "application/x-magic-cap-package-1.0"; break;
        case ".mcd": mimeType = "application/mcad"; break;
        case ".mcf": mimeType = "text/mcf"; break;
        case ".mcp": mimeType = "application/netmc"; break;
        case ".me": mimeType = "application/x-troff-me"; break;
        case ".mht": mimeType = "message/rfc822"; break;
        case ".mhtml": mimeType = "message/rfc822"; break;
        case ".mid": mimeType = "audio/midi"; break;
        case ".midi": mimeType = "audio/midi"; break;
        case ".mif": mimeType = "application/x-mif"; break;
        case ".mime": mimeType = "message/rfc822"; break;
        case ".mjf": mimeType = "audio/x-vnd.audioexplosion.mjuicemediafile"; break;
        case ".mjpg": mimeType = "video/x-motion-jpeg"; break;
        case ".mm": mimeType = "application/base64"; break;
        case ".mme": mimeType = "application/base64"; break;
        case ".mod": mimeType = "audio/mod"; break;
        case ".moov": mimeType = "video/quicktime"; break;
        case ".mov": mimeType = "video/quicktime"; break;
        case ".movie": mimeType = "video/x-sgi-movie"; break;
        case ".mp2": mimeType = "audio/mpeg"; break;
        case ".mp3": mimeType = "audio/mpeg"; break;
        case ".mpa": mimeType = "audio/mpeg"; break;
        case ".mpc": mimeType = "application/x-project"; break;
        case ".mpe": mimeType = "video/mpeg"; break;
        case ".mpeg": mimeType = "video/mpeg"; break;
        case ".mpg": mimeType = "video/mpeg"; break;
        case ".mpga": mimeType = "audio/mpeg"; break;
        case ".mpp": mimeType = "application/vnd.ms-project"; break;
        case ".mpt": mimeType = "application/vnd.ms-project"; break;
        case ".mpv": mimeType = "application/vnd.ms-project"; break;
        case ".mpx": mimeType = "application/vnd.ms-project"; break;
        case ".mrc": mimeType = "application/marc"; break;
        case ".ms": mimeType = "application/x-troff-ms"; break;
        case ".mv": mimeType = "video/x-sgi-movie"; break;
        case ".my": mimeType = "audio/make"; break;
        case ".mzz": mimeType = "application/x-vnd.audioexplosion.mzz"; break;
        case ".nap": mimeType = "image/naplps"; break;
        case ".naplps": mimeType = "image/naplps"; break;
        case ".nc": mimeType = "application/x-netcdf"; break;
        case ".ncm": mimeType = "application/vnd.nokia.configuration-message"; break;
        case ".nif": mimeType = "image/x-niff"; break;
        case ".niff": mimeType = "image/x-niff"; break;
        case ".nix": mimeType = "application/x-mix-transfer"; break;
        case ".nsc": mimeType = "application/x-conference"; break;
        case ".nvd": mimeType = "application/x-navidoc"; break;
        case ".o": mimeType = "application/octet-stream"; break;
        case ".oda": mimeType = "application/oda"; break;
        case ".omc": mimeType = "application/x-omc"; break;
        case ".omcd": mimeType = "application/x-omcdatamaker"; break;
        case ".omcr": mimeType = "application/x-omcregerator"; break;
        case ".p": mimeType = "text/x-pascal"; break;
        case ".p10": mimeType = "application/pkcs10"; break;
        case ".p12": mimeType = "application/pkcs-12"; break;
        case ".p7a": mimeType = "application/x-pkcs7-signature"; break;
        case ".p7c": mimeType = "application/pkcs7-mime"; break;
        case ".p7m": mimeType = "application/pkcs7-mime"; break;
        case ".p7r": mimeType = "application/x-pkcs7-certreqresp"; break;
        case ".p7s": mimeType = "application/pkcs7-signature"; break;
        case ".part": mimeType = "application/pro_eng"; break;
        case ".pas": mimeType = "text/pascal"; break;
        case ".pbm": mimeType = "image/x-portable-bitmap"; break;
        case ".pcl": mimeType = "application/vnd.hp-pcl"; break;
        case ".pct": mimeType = "image/x-pict"; break;
        case ".pcx": mimeType = "image/x-pcx"; break;
        case ".pdb": mimeType = "chemical/x-pdb"; break;
        case ".pdf": mimeType = "application/pdf"; break;
        case ".pfunk": mimeType = "audio/make"; break;
        case ".pgm": mimeType = "image/x-portable-greymap"; break;
        case ".pic": mimeType = "image/pict"; break;
        case ".pict": mimeType = "image/pict"; break;
        case ".pkg": mimeType = "application/x-newton-compatible-pkg"; break;
        case ".pko": mimeType = "application/vnd.ms-pki.pko"; break;
        case ".pl": mimeType = "text/plain"; break;
        case ".plx": mimeType = "application/x-pixclscript"; break;
        case ".pm": mimeType = "image/x-xpixmap"; break;
        case ".pm4": mimeType = "application/x-pagemaker"; break;
        case ".pm5": mimeType = "application/x-pagemaker"; break;
        case ".png": mimeType = "image/png"; break;
        case ".pnm": mimeType = "application/x-portable-anymap"; break;
        case ".pot": mimeType = "application/vnd.ms-powerpoint"; break;
        case ".pov": mimeType = "model/x-pov"; break;
        case ".ppa": mimeType = "application/vnd.ms-powerpoint"; break;
        case ".ppm": mimeType = "image/x-portable-pixmap"; break;
        case ".pps": mimeType = "application/vnd.ms-powerpoint"; break;
        case ".ppt": mimeType = "application/vnd.ms-powerpoint"; break;
        case ".ppz": mimeType = "application/vnd.ms-powerpoint"; break;
        case ".pre": mimeType = "application/x-freelance"; break;
        case ".prt": mimeType = "application/pro_eng"; break;
        case ".ps": mimeType = "application/postscript"; break;
        case ".psd": mimeType = "application/octet-stream"; break;
        case ".pvu": mimeType = "paleovu/x-pv"; break;
        case ".pwz": mimeType = "application/vnd.ms-powerpoint"; break;
        case ".py": mimeType = "text/x-script.phyton"; break;
        case ".pyc": mimeType = "applicaiton/x-bytecode.python"; break;
        case ".qcp": mimeType = "audio/vnd.qcelp"; break;
        case ".qd3": mimeType = "x-world/x-3dmf"; break;
        case ".qd3d": mimeType = "x-world/x-3dmf"; break;
        case ".qif": mimeType = "image/x-quicktime"; break;
        case ".qt": mimeType = "video/quicktime"; break;
        case ".qtc": mimeType = "video/x-qtc"; break;
        case ".qti": mimeType = "image/x-quicktime"; break;
        case ".qtif": mimeType = "image/x-quicktime"; break;
        case ".ra": mimeType = "audio/x-pn-realaudio"; break;
        case ".ram": mimeType = "audio/x-pn-realaudio"; break;
        case ".ras": mimeType = "application/x-cmu-raster"; break;
        case ".rast": mimeType = "image/cmu-raster"; break;
        case ".rexx": mimeType = "text/x-script.rexx"; break;
        case ".rf": mimeType = "image/vnd.rn-realflash"; break;
        case ".rgb": mimeType = "image/x-rgb"; break;
        case ".rm": mimeType = "application/vnd.rn-realmedia"; break;
        case ".rmi": mimeType = "audio/mid"; break;
        case ".rmm": mimeType = "audio/x-pn-realaudio"; break;
        case ".rmp": mimeType = "audio/x-pn-realaudio"; break;
        case ".rng": mimeType = "application/ringing-tones"; break;
        case ".rnx": mimeType = "application/vnd.rn-realplayer"; break;
        case ".roff": mimeType = "application/x-troff"; break;
        case ".rp": mimeType = "image/vnd.rn-realpix"; break;
        case ".rpm": mimeType = "audio/x-pn-realaudio-plugin"; break;
        case ".rt": mimeType = "text/richtext"; break;
        case ".rtf": mimeType = "text/richtext"; break;
        case ".rtx": mimeType = "text/richtext"; break;
        case ".rv": mimeType = "video/vnd.rn-realvideo"; break;
        case ".s": mimeType = "text/x-asm"; break;
        case ".s3m": mimeType = "audio/s3m"; break;
        case ".saveme": mimeType = "application/octet-stream"; break;
        case ".sbk": mimeType = "application/x-tbook"; break;
        case ".scm": mimeType = "application/x-lotusscreencam"; break;
        case ".sdml": mimeType = "text/plain"; break;
        case ".sdp": mimeType = "application/sdp"; break;
        case ".sdr": mimeType = "application/sounder"; break;
        case ".sea": mimeType = "application/sea"; break;
        case ".set": mimeType = "application/set"; break;
        case ".sgm": mimeType = "text/sgml"; break;
        case ".sgml": mimeType = "text/sgml"; break;
        case ".sh": mimeType = "application/x-sh"; break;
        case ".shar": mimeType = "application/x-shar"; break;
        case ".shtml": mimeType = "text/html"; break;
        case ".sid": mimeType = "audio/x-psid"; break;
        case ".sit": mimeType = "application/x-sit"; break;
        case ".skd": mimeType = "application/x-koan"; break;
        case ".skm": mimeType = "application/x-koan"; break;
        case ".skp": mimeType = "application/x-koan"; break;
        case ".skt": mimeType = "application/x-koan"; break;
        case ".sl": mimeType = "application/x-seelogo"; break;
        case ".smi": mimeType = "application/smil"; break;
        case ".smil": mimeType = "application/smil"; break;
        case ".snd": mimeType = "audio/basic"; break;
        case ".sol": mimeType = "application/solids"; break;
        case ".spc": mimeType = "text/x-speech"; break;
        case ".spl": mimeType = "application/futuresplash"; break;
        case ".spr": mimeType = "application/x-sprite"; break;
        case ".sprite": mimeType = "application/x-sprite"; break;
        case ".src": mimeType = "application/x-wais-source"; break;
        case ".ssi": mimeType = "text/x-server-parsed-html"; break;
        case ".ssm": mimeType = "application/streamingmedia"; break;
        case ".sst": mimeType = "application/vnd.ms-pki.certstore"; break;
        case ".step": mimeType = "application/step"; break;
        case ".stl": mimeType = "application/sla"; break;
        case ".stp": mimeType = "application/step"; break;
        case ".sv4cpio": mimeType = "application/x-sv4cpio"; break;
        case ".sv4crc": mimeType = "application/x-sv4crc"; break;
        case ".svf": mimeType = "image/vnd.dwg"; break;
        case ".svr": mimeType = "application/x-world"; break;
        case ".swf": mimeType = "application/x-shockwave-flash"; break;
        case ".t": mimeType = "application/x-troff"; break;
        case ".talk": mimeType = "text/x-speech"; break;
        case ".tar": mimeType = "application/x-tar"; break;
        case ".tbk": mimeType = "application/toolbook"; break;
        case ".tcl": mimeType = "application/x-tcl"; break;
        case ".tcsh": mimeType = "text/x-script.tcsh"; break;
        case ".tex": mimeType = "application/x-tex"; break;
        case ".texi": mimeType = "application/x-texinfo"; break;
        case ".texinfo": mimeType = "application/x-texinfo"; break;
        case ".text": mimeType = "text/plain"; break;
        case ".tgz": mimeType = "application/x-compressed"; break;
        case ".tif": mimeType = "image/tiff"; break;
        case ".tiff": mimeType = "image/tiff"; break;
        case ".tr": mimeType = "application/x-troff"; break;
        case ".tsi": mimeType = "audio/tsp-audio"; break;
        case ".tsp": mimeType = "application/dsptype"; break;
        case ".tsv": mimeType = "text/tab-separated-values"; break;
        case ".turbot": mimeType = "image/florian"; break;
        case ".txt": mimeType = "text/plain"; break;
        case ".uil": mimeType = "text/x-uil"; break;
        case ".uni": mimeType = "text/uri-list"; break;
        case ".unis": mimeType = "text/uri-list"; break;
        case ".unv": mimeType = "application/i-deas"; break;
        case ".uri": mimeType = "text/uri-list"; break;
        case ".uris": mimeType = "text/uri-list"; break;
        case ".ustar": mimeType = "application/x-ustar"; break;
        case ".uu": mimeType = "application/octet-stream"; break;
        case ".uue": mimeType = "text/x-uuencode"; break;
        case ".vcd": mimeType = "application/x-cdlink"; break;
        case ".vcs": mimeType = "text/x-vcalendar"; break;
        case ".vda": mimeType = "application/vda"; break;
        case ".vdo": mimeType = "video/vdo"; break;
        case ".vew": mimeType = "application/groupwise"; break;
        case ".viv": mimeType = "video/vivo"; break;
        case ".vivo": mimeType = "video/vivo"; break;
        case ".vmd": mimeType = "application/vocaltec-media-desc"; break;
        case ".vmf": mimeType = "application/vocaltec-media-file"; break;
        case ".voc": mimeType = "audio/voc"; break;
        case ".vos": mimeType = "video/vosaic"; break;
        case ".vox": mimeType = "audio/voxware"; break;
        case ".vqe": mimeType = "audio/x-twinvq-plugin"; break;
        case ".vqf": mimeType = "audio/x-twinvq"; break;
        case ".vql": mimeType = "audio/x-twinvq-plugin"; break;
        case ".vrml": mimeType = "application/x-vrml"; break;
        case ".vrt": mimeType = "x-world/x-vrt"; break;
        case ".vsd": mimeType = "application/x-visio"; break;
        case ".vst": mimeType = "application/x-visio"; break;
        case ".vsw": mimeType = "application/x-visio"; break;
        case ".w60": mimeType = "application/wordperfect6.0"; break;
        case ".w61": mimeType = "application/wordperfect6.1"; break;
        case ".w6w": mimeType = "application/msword"; break;
        case ".wav": mimeType = "audio/wav"; break;
        case ".wb1": mimeType = "application/x-qpro"; break;
        case ".wbmp": mimeType = "image/vnd.wap.wbmp"; break;
        case ".web": mimeType = "application/vnd.xara"; break;
        case ".wiz": mimeType = "application/msword"; break;
        case ".wk1": mimeType = "application/x-123"; break;
        case ".wmf": mimeType = "windows/metafile"; break;
        case ".wml": mimeType = "text/vnd.wap.wml"; break;
        case ".wmlc": mimeType = "application/vnd.wap.wmlc"; break;
        case ".wmls": mimeType = "text/vnd.wap.wmlscript"; break;
        case ".wmlsc": mimeType = "application/vnd.wap.wmlscriptc"; break;
        case ".word": mimeType = "application/msword"; break;
        case ".wp": mimeType = "application/wordperfect"; break;
        case ".wp5": mimeType = "application/wordperfect"; break;
        case ".wp6": mimeType = "application/wordperfect"; break;
        case ".wpd": mimeType = "application/wordperfect"; break;
        case ".wq1": mimeType = "application/x-lotus"; break;
        case ".wri": mimeType = "application/mswrite"; break;
        case ".wrl": mimeType = "application/x-world"; break;
        case ".wrz": mimeType = "x-world/x-vrml"; break;
        case ".wsc": mimeType = "text/scriplet"; break;
        case ".wsrc": mimeType = "application/x-wais-source"; break;
        case ".wtk": mimeType = "application/x-wintalk"; break;
        case ".xbm": mimeType = "image/x-xbitmap"; break;
        case ".xdr": mimeType = "video/x-amt-demorun"; break;
        case ".xgz": mimeType = "xgl/drawing"; break;
        case ".xif": mimeType = "image/vnd.xiff"; break;
        case ".xl": mimeType = "application/excel"; break;
        case ".xla": mimeType = "application/vnd.ms-excel"; break;
        case ".xlb": mimeType = "application/vnd.ms-excel"; break;
        case ".xlc": mimeType = "application/vnd.ms-excel"; break;
        case ".xld": mimeType = "application/vnd.ms-excel"; break;
        case ".xlk": mimeType = "application/vnd.ms-excel"; break;
        case ".xll": mimeType = "application/vnd.ms-excel"; break;
        case ".xlm": mimeType = "application/vnd.ms-excel"; break;
        case ".xls": mimeType = "application/vnd.ms-excel"; break;
        case ".xlt": mimeType = "application/vnd.ms-excel"; break;
        case ".xlv": mimeType = "application/vnd.ms-excel"; break;
        case ".xlw": mimeType = "application/vnd.ms-excel"; break;
        case ".xm": mimeType = "audio/xm"; break;
        case ".xml": mimeType = "application/xml"; break;
        case ".xmz": mimeType = "xgl/movie"; break;
        case ".xpix": mimeType = "application/x-vnd.ls-xpix"; break;
        case ".xpm": mimeType = "image/xpm"; break;
        case ".x-png": mimeType = "image/png"; break;
        case ".xsr": mimeType = "video/x-amt-showrun"; break;
        case ".xwd": mimeType = "image/x-xwd"; break;
        case ".xyz": mimeType = "chemical/x-pdb"; break;
        case ".z": mimeType = "application/x-compressed"; break;
        case ".zip": mimeType = "application/zip"; break;
        case ".zoo": mimeType = "application/octet-stream"; break;
        case ".zsh": mimeType = "text/x-script.zsh"; break;
        default: mimeType = "application/octet-stream"; break;
    }
    return mimeType;
} 

MS CRM 4.0 Attachment Size Limitation

MS CRM stores attachments directly in the database, and there is a 5Mb attachment limit by default with CRM 4.0. This limit is set for good reason, and if you want to increase it, you may want to see if it starts to impact your system performance.

This can be overwritten by doing the following:

  1. Changing the maxRequestLength value in the main CRM web.config file. By default it is 8Mb which doesn't impact the System Setting below. However if you increase the Maximum File size above the maxRequestLength you will have problems.

    <system.web>
        <httpRuntime executionTimeout="300" maxRequestLength="8192"/>
    </system.web>

    There are many examples on the web that show something like the following which pretty well removes any size constraints from the web.config.

    <httpRuntime executionTimeout="9999" maxRequestLength="99999"/>

  2. Save the web.config and restart IIS.
  3. For MS CRM 4.0 that just leaves adjusting the Maximum file size in System Settings with default of 5,120 Kb shown below to something higher but less than the maxRequestLength.

emailAtttachmentSize

Alternate 3. In MS CRM 3.0 there is a registry setting that needs to be set instead. In HKEY_Local_Machine\Software\Microsoft\MSCRM 
the maxuploadfilesize value can be changed from it's 5Mb default.

Sunday, October 12, 2008

CRM 4.0 Creating Activities

When importing activities or creating them for a process, there are three important Steps.

  1. Create the Activity relating it to all appropriate entities.
  2. (Optional) Attach a file using an Annotation or ActivityMimeAttachment
  3. Set the State of the Activity

MS CRM has numerous Activities. They are all part of an ActivityPointer Entity, but we won't work with that directly.  We will directly create task, email, letter, appointment, and phonecall entities. I'll cover Campaign and Service Activities another day.

Activities need to be related with a number of entities before they are useful.

Activities need

  1. An owner, a systemuser who is responsible for carrying an activity out.
  2. A regardingobject, the entity the activity is related to, like a Contact, Opportunity or Account.
  3. Numerous activityparty arrays for holding the different types of related entities that this activity affects. For example an email could cc: a systemuser and a contact, and there is a cc activityparty array for an email.
  4. Any file attachments in the form of an annotation or activitymimeattachment
  5. Start and End dates Scheduled and Actual.
  6. A state/status combination
    1. Active will show up in the owners WorkPlace Activities and under the regardingobject entity and possible rollup to an entity above that.
    2. Inactive - will show up in history under the regardingobject and possible rollup to an entity above that.

Setting up each activity is very similar. Below is some example code that creates a phonecall, and completes that activity.

Notice the following in the code sample:

  • The activityparty arrays that allow any number of systemuser, contact, account and lead entities to be related to an activity.
  • The SetStatePhoneCallRequest to set the state of the phonecall to completed. The PhoneCallStatus of 2 for outgoing. You can refer to the following link for this and find other state enumerations required to set the state and status of all entities. Link to MSDN PhoneCallState Enumeration
  • I use the CRM SDK CrmTypes and other Enumeration helpers. They come in handy so why not?
var newphoneCall = new phonecall
  {
      ownerid = CrmTypes.CreateOwner(EntityName.systemuser.ToString(), activityOwnerId),
      from = new[]
          {
              new activityparty
                  {
                      participationtypemask = CrmTypes.CreatePicklist(ActivityPartyType.Sender),
                      partyid = CrmTypes.CreateLookup(EntityName.systemuser.ToString(), activityOwnerId)
                  }
          },
      subject = subject,
      description = description,
      regardingobjectid = (matchedContact) ? CrmTypes.CreateLookup(EntityName.contact.ToString(), relatedContactId) : null,
      to =  (!matchedContact)? null : new []
         {
            new activityparty
              {
                  participationtypemask = CrmTypes.CreatePicklist(ActivityPartyType.ToRecipient),
                  partyid = CrmTypes.CreateLookup(EntityName.contact.ToString(), relatedContactId)
              }
         },
      scheduledstart = CrmTypes.CreateCrmDateTime(Convert.ToDateTime(scheduledStart).ToShortDateString()),
      actualstart = CrmTypes.CreateCrmDateTime(Convert.ToDateTime(scheduledStart).ToShortDateString()),
      scheduledend = CrmTypes.CreateCrmDateTime(Convert.ToDateTime(scheduledEnd).ToShortDateString()),
      actualend = (isComplete) ? CrmTypes.CreateCrmDateTime(Convert.ToDateTime(scheduledEnd).ToShortDateString()) : null,
  };

try
  {
      Guid phonecallId = service.Create(newphoneCall);

      if (isComplete)
      {
          var request = new SetStatePhoneCallRequest
              {
                  EntityId = phonecallId,
                   PhoneCallState = PhoneCallState.Completed,
                   PhoneCallStatus = 2
              };

          try
          {
              service.Execute(request);
          }

          catch (SoapException ex)
          {
              string errormsg = String.Format("Update Error Setting PhoneCall State to Complete error {0}", ex.Detail.InnerText);
              WriteErrorFile(errormsg);
          }
      }

  }
catch (SoapException ex)
   {
       string errormsg = String.Format("Write Error for PhoneCall {0} error {1}", row[(int) tblCal.ACTVCODE], ex.Detail.InnerText);
       WriteErrorFile(errormsg);
   }

Next: My next post will cover importing files into CRM in the form of annotations and activitymimeattachments as well as mimeTypes, and CRM file size limitations and their configuration.

Monday, October 6, 2008

MS CRM Data Import (Cleaning up the Mess)

 

Before importing data into CRM it, I think it is important to know how to get it back out, how long it will take to get it back out, or how to get back to a clean state.

Why is that?

1. Time Estimation!  Creating a CRM record is FAST. Deleting a CRM record can be VERY slow!  There is a reason for this. When you create a new Account, there is a new record added to a Base and Extension base table. That's pretty simple! When you delete an Account, CRM checks for all of the dependencies and potentially deletes a number of child records in other tables, verifies that there are no records that restrict the deletion of a parent and finally deletes the Account.

To drive this home, using my import framework, I recently imported about 34,000 contacts from an Access database into a CRM system running on a VPC image on my laptop. It took 24 minutes to import these contacts, while updating associated accounts with primary contacts where applicable. However it took about 8 hours just to delete these Contacts! Granted, a VPC image running off of a single core of a laptop is going to be very slow in comparison to most production systems, but many imports are MUCH larger than 34,000 contacts.

2. Some data doesn't want to be deleted! Some CRM entities end up in a state where the CRM web service can not delete them.  Let's say that you import a company's Contracts with the expiration dates set. MS CRM being the smart system that it is will start changing the state of all contracts with a past expiration date to "Expired". This is a neat feature, but now you can not delete that entity. In fact you can not even set the state of that contract back to draft using web services so that you can delete it. There is a work around for this, but it is unsupported and requires using SQL.

3. Data must be removed in sequence! Because there are lots of relationships between entities in CRM, removing data typically means deleting data in the reverse order of your imports.  So that the last things that you can typically delete are the Contacts and then the Accounts. You must delete Quotes, Orders, and Invoices before the related Opportunities, etc. You must be aware of the data's hierarchy.

4. The Deletion Service. When you delete an entity using the CRM web service, the entity is being marked with a flag to be deleted after the Deletion Service runs. With MS CRM 4.0 that happens once a day by default, however you can use the CRM 4 Scale Group Job Editor to adjust that. With CRM 3.0 the default was every two hours and it could be adjusted with a registry setting. Knowing when this service runs is important if you want to back up a clean database.

Best Practices - Data Import Cleanup

Strong Suggestion! Use a Test system or VPC image for data import testing. Some of the methods I am going to cover for data cleanup are unsupported by MS. It is much better than you do all of your import testing on something other than the production CRM system.

The Quick Reset to Ground Zero

  1. When your base configuration and data are clean, and the Deletion service has finished running, back up the database pair MSCRM_CONFIG, and <BaseBUName>_MSCRM to a safe place.
  2. When you need to clean out your imported data, export your Customizations to a safe area then restore both databases and import and publish your customizations.

 

This is a great way to do things as long as you can keep track of any configuration changes between resets. However sometimes you may have hours invested in getting everything imported to a certain state and you just need to remove just the last thing that you imported and it would take more time to reset your system and import everything up to that point.

Proceed at your own risk! To reset your database bringup SQL Server Management Studio under Management-> activityMonitor to find the processes blocking your CRM database. Stop all of the CRM Services and IIS, then refresh the activityMonitor to see what is left. If there is anything left, then kill that process and when the activityMonitor is clear of references to your databases you can restore like below.

Alter Database MicrosoftCRM_MSCRM
  SET SINGLE_USER With ROLLBACK IMMEDIATE 

  RESTORE DATABASE MicrosoftCRM_MSCRM
  FROM DISK = 'C:\MicrosoftCRM_MSCRM.bak'

use Master 
Alter Database MicrosoftCRM_MSCRM 
  SET MULTI_USER WITH ROLLBACK IMMEDIATE;
Make sure to also do this for your CONFIG database. This will most likely require a reboot when you are finished.
Alter Database MSCRM_CONFIG
  SET SINGLE_USER With ROLLBACK IMMEDIATE 

  RESTORE DATABASE  MSCRM_CONFIG 
  FROM DISK = 'C:\MSCRM_CONFIG.bak'
  WITH REPLACE

use Master 
Alter Database MSCRM_CONFIG 
  SET MULTI_USER WITH ROLLBACK IMMEDIATE;
The Surgical Cleanup

This is when knowledge of the relationships between your entities and the requirements to put an entity into a state that allows deletion are important.

Example:  Cleaning up Contracts

Below you can see the BackgroundWorker DoWork method called from my CRM Import Winform Application. Every Import has a "Delete All" button, and a progress bar to allow an easy check the progress by glancing over at it.

        /// <summary>
        /// Delete Entity
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="e"></param>
        private void deleteWorker_DoWork(object sender, DoWorkEventArgs e)
        {
            var arg = (ImportArguments)e.Argument;

            var crmService = new webserviceHelper(arg.errorFile, arg.logFile);

            // Returns a collection with only the entity's id to use for deletion
            BusinessEntityCollection deleteCollection = crmService.GetEntitiesForDelete(EntityName.contract.ToString(), "contractid");

            float progressCount = 0;
            float progressTotal = (float) deleteCollection.BusinessEntities.Length;

            var directAccess = new crmdb();
            
            // Sets all Contracts into a Draft state and clear Originating Contract.
            directAccess.ClearContractsforDelete();

            foreach (contract deleteEntity in deleteCollection.BusinessEntities)
            {
                
                if (deleteWorker.CancellationPending)
                {
                    return;
                }

                // Update ProgressBar
                progressCount++;
                deleteWorker.ReportProgress((int)((progressCount / progressTotal) * 100));
               
                try
                {
                  crmService.service.Delete(entity, deleteEntity.contractid.Value);
                }

                catch(SoapException ex)
                {
                    MessageBox.Show(ex.Detail.InnerXml, string.Format("Delete {0}s Failed", entity), MessageBoxButtons.OK, MessageBoxIcon.Error);
                }
            }
Below is the SQL that is called to remove references to any Originating Contracts, and to put the Contract in a Draft state so that it can be deleted.
        /// <summary>
        /// PutsContract in a Draft state so it can not be deleted.
        /// NOTE: This is temporary as CRM will periodically start to mark contracts as expired!
        /// </summary>
        public void ClearContractsforDelete()
        {
            using (var connection = new SqlConnection(crmConnectionString))
            {
                connection.Open();

                string command = string.Format(
                    @"UPDATE dbo.ContractBase
                        SET 
                        StateCode = 0      
                        ,StatusCode = Null      
                        ,OriginatingContract = Null "
                );

                var cmd = new SqlCommand(command, connection);
                cmd.ExecuteNonQuery();
            }
        }

There is very similar logic for invoices, and quotes. Other entities like opportunities can be re-opened and then deleted.

Wednesday, October 1, 2008

Importing Data into MS CRM - Overview

MS CRM is a great place to permanently consolidate data silos, which could be in Excel, Access, external SQL databases or any other data source, but there are also many useful reasons for bringing data into CRM periodically or just having CRM accessing external data stores.

Integrating MS CRM with the rest of the world is a passion of mine and pulling data into CRM, driving CRM with external data and sharing data from CRM to drive external processes is key to powerful integration.

I put CRM data exchanges into a number categories.

  1. One Shot: Initial Imports from systems replaced by CRM.
  2. Scheduled Imports: Ongoing data pulled from another system.
  3. Synchronization: Synchronization with an external database.
  4. Business Process drivers: External data is consulted to start business processes within CRM.
  5. Integration: CRM and another application share data with each other.
  6. External Control: CRM drives external processes

What I would like to do is examine these topics and share some of the tools and parts of my import framework over the next few months.

I think that this is a good point to mention Scribe. I have used Scribe, but none of my tools or examples will be for Scribe.  Scribe has an excellent reputation and is a valuable tool for many. They also offer numerous connectors for moving data between CRM and other mainstream applications. I highly recommend Scribe if you are not a developer familiar with the CRM SDK.

Since I consider myself a .NET developer first, I have found that working with the SDK gives me a lot more low level control and flexibility, and none of my code or methods will require the purchase of a 3rd party tool.

Development side note: While developing my data import framework which uses CRM web services, I had to work with MS to find out why my application kept crashing.  It turned out that it was importing records so quickly that the web services were overloading the TCP/IP Sockets on the W2003 Server. The fix was to modify the registry settings to both drastically increase the number of Sockets available and to reduce the recycle time that that Sockets would sit idle before being reclaimed for reuse. This may not have been a big deal in the grand scheme of things, but that didn't stop me from having a brief Tim Allen moment when I found out that I completely blew by the default settings for W2003 :)

Saturday, September 20, 2008

Hint: Mapping Data Tables for CRM Imports

When importing data into a CRM database, there is an easy way to create current mapping tables.

First take the url for your CRM server and append /sdk/list.aspx to it.
example.  http://crmserver1:5555/sdk/list.aspx

That will give you a current listing of all of your entities with customizations.
Then click a particular appropriate entity to get a screen like the one below.

entityfieldbrowser

That's very informative, but we want to re-purpose this for use in a data import mapping table.

  1. Right click -> View Source
  2. Ctrl A  ( select all of the raw HTML )
  3. Ctrl C  ( copy it to your clipboard )
  4. Now open up excel. Excel is really accepting of a lot of data formats these days and most customers like to work in Excel Spreadsheets
  5. Paste that RAW HTML into a spreadsheet
  6. Do a little formatting and remove what you don't need.

Now you have CRM Entity import tables reflecting any customizations that you have made.

excel_entitymap