MS CRM 2013 Online upgrade to Office 365 authentication

Recently many CRM Online customers were transitioned from Live ID to Office 365 authentication.

The result is that the earlier Live ID users accounts no longer exist and any software still using Live ID authentication stopped working.

This requires updating how any portal code authenticates and using a new Office 365 user account to authenticate with.

In addition the CRM 2013 Email Router needs to be reconfigured and updated to the Rollup 2 version ( at this moment in time ).

O365 Authentication

If you start with the latest CRM 2013 SDK  example AuthenticateWithNoHelp solution, you can add the code snippet below to the file  AuthenticateWithNoHelp.cs to get O365 authentication to work.

First open and compile this solution. Than add the code snippet below to the AuthenticationWithNoHelp class.

The following gives you an IOrganizationService and an IOrganizationServiceContext

If you are incorporating this into an existing CRM 2011 solution, you will need to update your Microsoft.Xrm.Client and Microsoft.Xrm.Sdk references.

The global variables beginning with underscores are used internally in the AuthenticationWithNoHelp class. Substitute your organization name for {orgname}.

public IOrganizationService OrgService { get; set; }
public OrganizationServiceContext OrgContext { get; set; }

private String _discoveryServiceAddress = "https://dev.crm.dynamics.com/XRMServices/2011/Discovery.svc";
private String _userName = "newusername@{orgname}.onmicrosoft.com";
private String _password = "password";
private String _domain = ""; 


public Authenticate()
{
    IServiceManagement<IDiscoveryService> serviceManagement = ServiceConfigurationFactory.CreateManagement<IDiscoveryService>(new Uri(_discoveryServiceAddress));
    AuthenticationProviderType endpointType = serviceManagement.AuthenticationType;
    
    IServiceManagement<IOrganizationService> orgServiceManagement = ServiceConfigurationFactory.
        CreateManagement<IOrganizationService>( new Uri("https://{orgname}.api.crm.dynamics.com/XrmServices/2011/Organization.svc "));

    AuthenticationCredentials credentials = GetCredentials(orgServiceManagement, endpointType);
    
    using (OrganizationServiceProxy organizationProxy =
        GetProxy<IOrganizationService, OrganizationServiceProxy>(orgServiceManagement, credentials))
    {
        OrgService = organizationProxy;

        organizationProxy.EnableProxyTypes();
     
        OrgContext = new OrganizationServiceContext(organizationProxy);
    }
}

Email Router

If you are running an older version of the email router update to the appropriate 32 or 64 bit version found here.

http://www.microsoft.com/en-us/download/details.aspx?id=42272
Install and reboot.

 

To reconfigure your Email Router:

1. Change the Server URL  from dev.crm.dynamics.com to disco.crm.dynamics.com

2. Change the User Name and Password to new a new Office 365 account

Then under Configuration Profiles Set the Outgoing Access Credentials to match the Office 365 Credentials used above.

Now

1. Publish your changes
2. Load Data
3. Test Access

That may not cover every Email Router configuration change, but it should cover most of them.

MS CRM and Portals (fast caching method)

Let’s say that you have a portal application that is very data driven by highly related data, or that has some displays that require a lot of heavily related data to very flexibly dynamically build the controls on the pages.  Leveraging the quick prototyping of MS CRM to design your schema, and managing this controlling data within  MS CRM helps abstract away a lot of grunt work and very quickly get  your portal online,  but the time required to retrieve a long stream of related data by web services is affecting the performance of your website. Because your pages are very data driven depending on the user who logs in some of the website caching available out of the box isn’t helping much because of many web services calls required to customized the information.

What if you could cache all of this related data that you needed in an extremely quick and efficient way so that the portal could get the data that drives it in an nearly instant way?

In this example there is a MS CRM Online system and an interactive Dashboard displayed on a Portal. This is a simple example, but the real project contains a very large amount of data nested many levels deep.  All the queries by web services would take well over a minute or two to run, but a typical portal user is not that patient.

Caching Mechanism

Advantages of this mechanism:

1. Requires a single select to a local database to fetch extremely complex relational data that is already organized into a consumable format.

2. Utilizes a background task to do the heavy lifting to keep things up to date without the portal user having to wait for a many web services calls to MS CRM and also allows for processing of the raw data if need be.

3. Can be part of a plan to allow a portal to be self sufficient for periods of time when the CRM system is not available.

Overview

Part 1: Filling the Cache

1. Query data from MS  CRM via web services in a background task. This could be a scheduled applications or service on a timer that keeps the cached updated.
2. Put data into related serializable containers under one container with all of the information needed for a single account in the format needed to drive your application.
3. Serialize this container
4. Write it to the local SQL database as a single blob with an identifier.

Part 2: Reading the Cache:

1. With a single select from the local database return the entire blob of information needed.
2. Deserialize this blob
3. Cast it to the Parent container class.
4. Consume it in your Portal application.

Example Code 

This example is very simplistic. The class structure and queries to pull MS CRM data can be as complex as needed. The real gains to using this are when there is a lot of related data requiring a large number of queries.

Nested Serialized Classes

All classes must be serializable. The top level container class will contain all other containers or lists of containers

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Linq;
using System.Web;
using System.Xml;
using System.Xml.Serialization;


/// <summary>
/// Top Level Serializable Container 
/// All classes contained must also be serializable 
/// </summary>
[Serializable]
public class DashboardBlob
{
    public Guid AccountId;
    public DateTime LastUpdated;
    public List<ContactS> students = new List<ContactS>();
}

/// <summary>
/// Nested Serializable class
/// </summary>
[Serializable]
public class ContactS
{
    public Guid ContactId;
    public string FirstName;
    public string LastName;
    public string EMailAddress1;
}

Filling the Serialized Classes

This can be done in any way that queries the data needed and fills the appropriate serialized classes.

/// <summary>
/// Create Top level container and call methods to fill all nested containers
/// </summary>
/// <param name="accountId"></param>
public static DashboardBlob FillDashboardBlob(Guid accountId)
{
    var blob = new DashboardBlob
        {
            AccountId = accountId,
            LastUpdated = DateTime.Now,
            students = GetSerializedContactsByAccount(accountId)
            //pull all other related data and nested data.
        };

    return blob;
}

/// <summary>
/// Query that pulls data from a CRM context and stores it in a serializable class
/// </summary>
/// <param name="accountId"></param>
/// <returns></returns>
public static List<ContactS> GetSerializedContactsByAccount(Guid accountId)
{
    var serializedContacts = 
        (from c in DemoContext.ContactSet
        join students in DemoContext.new_account_studentSet on c.ContactId equals students.contactid
        where students.new_accountid == accountId
        orderby c.LastName, c.FirstName
        select new ContactS
        {
            ContactId = c.ContactId.Value,
            FirstName = c.FirstName,
            LastName = c.LastName,
            EMailAddress1 = c.EMailAddress1
        }).ToList();

    return serializedContacts;
}

The code below this point is nearly generic with the exception of the field names for your SQL tables.

Create a simple local SQL table for cache storage

Save/Update the Local SQL Cache

Now that you have the data stored in a serialized container, serialize that container and store in the local SQL server database as a blob.

/// <summary>
/// Update that writes the blob back to SQL
/// </summary>
/// <param name="blob"></param>
/// <param name="connectionString"></param>
/// <param name="isCreate"></param>
public static void SaveUpdateDashboardBlob(DashboardBlob blob, string connectionString, bool isCreate)
{
    var bytes = SerializeAnObject(blob);
    var connection = new SqlConnection(connectionString);
    SqlCommand sqlCmd;
    connection.Open();
    if( isCreate) 
        sqlCmd = new SqlCommand("INSERT INTO dbo.MainCache(accountid,updated,dashboardxml) VALUES (@id,@updated,@xml)", connection);
    else
        sqlCmd = new SqlCommand("UPDATE dbo.MainCache Set updated=@updated, dashboardxml=@xml where accountid=@id", connection);
    sqlCmd.Parameters.Add("@id", SqlDbType.UniqueIdentifier);
    sqlCmd.Parameters["@id"].Value = blob.AccountId;
    sqlCmd.Parameters.Add("@updated", SqlDbType.DateTime);
    sqlCmd.Parameters["@updated"].Value = blob.LastUpdated;
    sqlCmd.Parameters.Add("@xml", SqlDbType.Xml, Int32.MaxValue);
    sqlCmd.Parameters["@xml"].Value = bytes;
    sqlCmd.ExecuteNonQuery();
    connection.Close();
}

public static string SerializeAnObject(object AnObject)
{
    var Xml_Serializer = new XmlSerializer(AnObject.GetType());
    var Writer = new StringWriter();
    Xml_Serializer.Serialize(Writer, AnObject);
    return Writer.ToString();
}

Read the Local SQL Cache

This is the payoff. A single call to the local database returns a completely organized sorted and ready for consumption data container.

/// <summary>
/// Read the blob from local SQL database, deserialize it and cast it back to the container class.
/// </summary>
/// <param name="accountId"></param>
/// <param name="connectionString"></param>
/// <returns></returns>
public static DashboardBlob GetDashboardBlob(Guid accountId, string connectionString)
{
    var connection = new SqlConnection(connectionString);
    connection.Open();
    var adapter = new SqlDataAdapter(string.Format("select dashboardxml from dbo.MainCache where accountid='{0}' ", accountId), connection);
    var ds = new DataSet();
    adapter.Fill(ds);
    DataTable table = ds.Tables[0];
    connection.Close();

    if (table.Rows.Count != 0)
    {
        var data = (DashboardBlob)DeSerializeAnObject(table.Rows[0][0].ToString(), typeof(DashboardBlob));
        return data;
    }
    return null;
}

public static Object DeSerializeAnObject(string xmlOfAnObject, Type objectType)
{
    var strReader = new StringReader(xmlOfAnObject);
    var xmlSerializer = new XmlSerializer(objectType);
    var xmlReader = new XmlTextReader(strReader);
    try
    {
        Object anObject = xmlSerializer.Deserialize(xmlReader);
        return anObject;
    }
    finally
    {
        xmlReader.Close();
        strReader.Close();
    }
}

The new MS CRM 2011 64 bit VM and VirtualBox

I’ve gotten a number of emails asking what was needed to get the new Partner Source MS CRM 2011 64bit Virtual Machine to work in Virtual Box in Windows 7.

The compatibility issue appears to be the default hard disk settings. The following is a workaround that has worked for many people.  I’ve been told this doesn’t work on Windows 2008 Servers, but on a 2008 server you could also run Hyper-V.

Change your VHD to run from an IDE controller:

Open up your storage settings and add the vhd as an IDE drive.  Click the right + icon to create a new virtual hard  drive. Choose an existing drive.

Select the VHD file the you decompressed and open it.

Now  remove your SATA drive

Now Click OK.

Memory

This VM likes 2.5Gb or more RAM to operate in. As always more is better especially if you are going to demonstrate both CRM and SharePoint together.

Now Start it up!

Addendum

You should enable hardware virtualization when running 64-bit OS’s.  This  is a firmware setting on your motherboard. This is normally called “Intel Virtualization”,  “VT-x” or “Intel VT” and AMD has an equivalent setting. Laptops frequently have this disabled by default.

CRM 2011 OData, JSON and CRM Forms

1. Generating OData queries

OData commands can easily be tested out in IE. To help with this, you will want to turn off the feed reading view in IE.

The first thing that you will want is the name of the Sets that you will be calling on. If you use a path like the following it will provide a listing of the names of your sets. http://crmdev:5555/CRMDEV/XRMServices/2011/OrganizationData.svc

You will see a list of them in the following format.

- <collection href="AccountSet">
  <atom:title>AccountSet</atom:title> 
  </collection>

Since these names are case sensitive you will want to look at the names of your custom entities. Your stock items like AccountSet and ContactSet will be camel cased, but your custom entities will likely show up as new_myentitySet.

http://crmdev:5555/CRMDEV/XRMServices/2011/OrganizationData.svc/AccountSet

As you can see the field names use the Schema Name in CRM rather than the lowercase name the is used on the CRM forms. This is something that can cause confusion.

<d:Address1_Name m:null="true" />
<d:Address1_Telephone2 m:null="true" />
<d:OverriddenCreatedOn m:type="Edm.DateTime" m:null="true" />
<d:Telephone3 m:null="true" />
<d:DoNotBulkPostalMail m:type="Edm.Boolean">false</d:DoNotBulkPostalMail>

If you specify the guid of that entity, the results returned will be for that one entity. All fields are returned whether they are null or not. This listing will show you the exact case of each attribute name that you may want to query.

http://crmdev:5555/CRMDEV/XRMServices/2011/OrganizationData.svc/AccountSet(guid'3D37ACD3-5846-E011-8AE1-080027131692')

You can save bandwidth by only selecting the fields that you need and only those fields will be returned. Below will only return the AccountNumber and AccountName

http://crmdev:5555/CRMDEV/XRMServices/2011/OrganizationData.svc/AccountSet(guid'{3D37ACD3-5846-E011-8AE1-080027131692}')?$select=AccountNumber,AccountName

There are numerous references from MS explaining how to build OData Queries, however, Rhett Clinton’s recent addition to Codeplex is probably the easiest way to generate these queries. His tool can be found at the link below.

http://crm2011odatatool.codeplex.com/

Screen shot of his Query Designer

2. Using OData Queries with JSON

To use any of the following in a javascript library as webresource in CRM 2011 solution, you will first need to include a jquery library and a json library. The jquery1.4.1.min.js and json2.js files can be found in the most recent MS CRM SDK. sdk\samplecode\js\restendpoint\jqueryrestdataoperations\jqueryrestdataoperations\scripts. Add these a libraries and include them above the JavaScript library that you put your code in. Click here to see what that looks like in the Forms Property page.

To utilize these OData queries in a CRM Form using JSON, there is a pretty standard template to use. Just set the odataSelect below to your OData select url, and use the appropriate return method.

var odataSelect = "Your OData Query";

$.ajax({
       type: "GET",
       contentType: "application/json; charset=utf-8",
       datatype: "json",
       url: odataSelect,
       beforeSend: function (XMLHttpRequest) { XMLHttpRequest.setRequestHeader("Accept", "application/json"); },
       success: function (data, textStatus, XmlHttpRequest) 
           { 
               // Use only one of these two methods

               // Use for a selection that may return multiple entities
               ProcessReturnedEntities(data.d.results); 

               // Use for a single selected entity
               ProcessReturnedEntity(data.d);
           
           },
       error: function (XmlHttpRequest, textStatus, errorThrown) { alert('OData Select Failed: ' + odataSelect); }
   });

Notice the two methods:

• ProcessReturnedEntities(data.d.results)
• ProcessReturnedEntity(data.d)

When selecting what could be any number of entities, there will be an array returned, and you want to look at the data.d.results.  When selecting a specific Guid there is no results array created, and you will need to just look at the data.d that is returned.

For example:

function ProcessReturnedEntities(ManyEntities)
{
  for( i=0; i< ManyEntities.length; i++)
  {
     var oneEntity = ManyEntities[i];
     var accountNumberAttribute = oneEntity.AccountNumber;    
     
     var accountNumberValue = eval(oneEntity.AccountNumber);
  }
}

function ProcessReturnedEntity(OneEntity)
{  
     var oneEntity = OneEntity;
     var accountNumber = oneEntity.AccountNumber;
     
     var accountNumberValue = eval(oneEntity.AccountNumber);     
}

Entity Attributes

As you can see the JavaScript entity objects returned have attributes named with matching camel casing in the OData query.

In that case of simple CRM attributes like:

• string
• memo
• decimal
• double
• integer

You can get the value from them by simply using eval like shown in the example above.

For the following CRM attributes there is more involved.

• optionset
• money
• datetime
• lookup

For example:

var moneyValue = eval( oneEntity.new_MoneyAttribute.Value);
var optionSetValue = eval ( oneEntity.new_OptionSet.Value);

 

Setting CRM Form Fields with Queried Values

This gets a bit more complex when setting values to CRM form controls.

1. The form field names are all lower case, so the retrieved names do not match.
2. The form fields have validation and maintain more types than the returned OData values have.
3. There is some conversion required between them.

You can find out the type of a form control as follows:

var attrType = Xrm.Page.getAttribute("accountnumber").getAttributeType();

With the type you can then use the appropriate means to set form controls.

string, memo fields:

Xrm.Page.getAttribute("accountnumber").setValue(eval(oneEntity.AccountNumber));

decimal, double fields:

Xrm.Page.getAttribute("new_float").setValue(parseFloat(eval(oneEntity.new_Float)));

integer fields

Xrm.Page.getAttribute("new_integer").setValue(parseInt(eval(oneEntity.new_Integer)));

money fields

Xrm.Page.getAttribute("new_moneyattribute").setValue(parseFloat(eval(oneEntity.new_MoneyAttribute.Value)));

optionset fields

Xrm.Page.getAttribute("new_optionset").setValue(eval(oneEntity.new_OptionSet.Value));

date fields

var fieldValue = eval(oneEntity.new_DateTime);                       
var dateValue = new Date(parseInt(fieldValue.replace("/Date(", "").replace(")/", ""), 10));
Xrm.Page.getAttribute("new_datetime").setValue(dateValue);

The addition of support for JSON, OData queries in MS CRM 2011 has

added some great power to what can be done easily in the JavaScript of a CRM form. This should reduce the number of solutions that require web applications running in iframes dramatically. This is a good thing since MS is phasing out the ISV folder for web applications running in the same app pool on a web server and there is currently no support for aspx files in CRM solutions.

CRM 2011 Set Default Transaction Currency in JavaScript

The following function uses JSON and OData to select the first Currency entity that is configured on your system and then sets the transactioncurrencyid lookup to be that currency.

This particular feature is just as relevant for CRM 2011 as it was for CRM 4.0, but it is easier to implement now.

The OData selection for this consists of the following:
/TransactionCurrencySet?$select=TransactionCurrencyId,CurrencyName

If you have multiple currencies, you can always add a filter to OData select to grab the specific currency that you need. This function uses the first currency returned.

function SetDefaultCurrency()
{
    var context = Xrm.Page.context;
    var serverUrl = context.getServerUrl();
    var ODataPath = serverUrl + "/XRMServices/2011/OrganizationData.svc";   
    var odataSelect = ODataPath + "/TransactionCurrencySet?$select=TransactionCurrencyId,CurrencyName";

    $.ajax({
        type: "GET",
        contentType: "application/json; charset=utf-8",
        datatype: "json",
        url: odataSelect,
        beforeSend: function (XMLHttpRequest) { XMLHttpRequest.setRequestHeader("Accept", "application/json"); },
        success: function (data, textStatus, XmlHttpRequest)
        {            
            var myCurrency = data.d.results[0];
            var idValue = eval('myCurrency.TransactionCurrencyId');
            var textValue = eval('myCurrency.CurrencyName');    
            var thisEntityType = 'transactioncurrency';            
            Xrm.Page.getAttribute("transactioncurrencyid").setValue([{ id: idValue, name: textValue, entityType: thisEntityType }]);         
        },
        error: function (XmlHttpRequest, textStatus, errorThrown) { alert('OData Select Failed: ' + odataSelect); }
    });
}

In order to use this. You need to make sure that you include jquery1.4.1.min.js and json2.js as libraries first since this script relies on them. Create another library with the above function. If you just call the SetDefaultCurrency function in the OnLoad event of your form, the currency will be set as the form opens and will be be ready for you to set any currency values that rely on this.

The jquery1.4.1.min.js and json.js files can be found in the most recent MS CRM SDK. sdk\samplecode\js\restendpoint\jqueryrestdataoperations\jqueryrestdataoperations\scripts

CRM 2011 RTM VM Installation Differences

There are a few differences between the RTM installation and the Beta installation I have documented in earlier blog articles. Starting with the following: http://crmscape.blogspot.com/2010/09/creating-ms-crm-2011-vm-part-1-of-2.html

1. There are minor appearance changes in the CRM 2011 Installation screens, but everything else looks close enough that the Beta instructions still work.

2. The entire section of my instructions called SQL Reporting Services ( Configure Data Source ) is unnecessary, which is an improvement.  However I did add my Admin user account to the PrivUserGroup and PrivReportingGroup and then everything worked great.

Otherwise the instructions still worked well for me. I’ll look over the updated installations instructions shortly to make sure there isn’t anything of importance that needs to be added.

I’m also using the latest Virtual Box 4.0.4 release for this which is working well. The only reason I used it instead of VMWare is that I had an existing VM just waiting for MS CRM to be installed with all the prerequisites in place.

I’ve been “living on a cloud” for the past few months in CRM Online and Windows/SQL Azure land, but now that the 2011 On Premises release is out, I’m back on terra firma working with that again too.

MS CRM 2011, the “Cloud”, Azure and the future

Something really important is happening right now, and if you are involved in MS CRM take notice or fall by the wayside!  This is a real game changer!

Almost all of my MS CRM installations were on premises installations until just this Fall.  Sure there were IFD configurations and web portals with Silverlight etc., but the customer was always in tight control of the servers, even if the servers were hosted.

I thought the next big thing would be the tighter SharePoint 2010 integration with MS CRM 2011 and to be fair I have seen interest in that and it will be a growing thing.

What I didn’t expect is that suddenly I would almost exclusively be working with hosted MS CRM systems, MS CRM Online and Windows Azure and SQL Azure. BTW I’ll be working with the new SQL Azure Reporting Services CTP this week as well.

What is pretty obvious is that MS has a lot of technologies coming together at about the time that MS CRM 2011 is hitting the scene. Frankly I thought it all looked interesting, but I didn’t expect the market to react so quickly or for it all to fit together so well.

What I also didn’t expect was how much I would enjoy this transfer of responsibility. I enjoy being in control of things, but I also love abstracting away busy work to concentrate on solving real problems.  That is the point of MS CRM. System administration work is not where the fun is, at least for me.

I was also pleasantly surprised at how easy using SQL Azure is. Just make sure you are using SQL 2008 R2 Management Studio and it is really nothing more than setting up an IP range you will be connecting through in the Azure portal, and changing your connection string. All of your tables need to have Clustered indexes as well and you have to use SQL scripts for all of your modifications, but otherwise it is just like the SQL you already know. Another thing to beware of is that the Web database is 1-5Gb and if you expect to grow beyond that you need to start with a Business Database which starts at 10Gb and can grow to 50Gb.

I’m currently working on a project that is using an Azure 10Gb database, the new to SQL 2008 geography fields and spatial indexes. I imported about 7Gb of data in about 3.5 hours. The database has been very responsive.  I did notice that when I ran a very intensive task like generating a Spatial index across 9.2 million records that it would occasionally idle my process, but it still finished in a respectable time.

Windows Azure, likewise, was very easy to set up.  Install the Azure SDK, create a Cloud Service Project, setup the Diagnostics and build a web application. If you make the Cloud Service your Start project it can be debugged locally as it will be deployed. There are a few configuration switches, but then again you are abstracting away all of the IIS administration.

Where this all gets interesting is that additional instances of your application can be added with a few key clicks and without purchasing equipment or understanding anything about network load balancing. Adjust the amount of CPU cores and memory your application needs as it grows.  Scale it back during the off season or grow though the ceiling.

I have customers who want to scale up quickly to compete with some pretty large competitors with a better product, but without the infrastructure or small army to maintain it. The Azure platform allows a small guy with a better mousetrap to take on the world.

As a lone gun for hire, without a staff of sysadmins, what this means to me is that I can create some really really big solutions very quickly and without the setup time normally involved, or the meetings with IT departments to budget for servers, and schedule for installation and configuration.

What this also is going to do is let a lot more killer apps actually see the light of day. I think we are going to see a new wave of enabled developers make a lot of really cool things happen.

The cost of admission to see if something will stick has really dropped and if your solution is really that good and your website is nailed with traffic, all you need is a few key clicks to give you more bandwidth, memory and processing power.

I was a bit skeptical at first, but as the saying goes, “People vote with their wallets”.  Based on what I am seeing the people are voting and much quicker than I expected!