Azure ExpressRoute for Office 365

This is going to be a quick post, but hopefully it can help someone from getting the same unpleasant surprise as I recently encountered.   If you’re working on a large scale Office 365 implementation, you may have had your interest piqued by their announcement of an Azure ExpressRoute option for Office 365 in late  2015.  This can be a pretty big performance win if the organization in question already has a significant investment in WAN infrastructure like an MPLS cloud.

Hopefully I’ll have more detail on why this is once we get a better explanation, but if you’re planning on this and pricing it out make sure you plan for the premium add-on.  I recently got well down the road to implementing this, where the expected amount of Azure service credit was already purchased, only to be informed while provisioning circuits that this was a requirement for use with Office 365.

There are probably good reasons why this is the case, but an extra $36,000/year/circuit is not exactly a pleasant surprise.  If anyone that has contacts at Microsoft happens to see this, please please please have someone update your planning docs.   Also if I happen to just be overlooking the really obvious place where this is already documented, I’d love to get a link in the comments or something.  I’d be happy to update the post.

Kerberos Single Sign-On in iOS

Update: Just a quick note prompted by a comment.  These steps should still work in iOS 8 releases, although I have not updated the post to talk about any of the new capabilities related to certificates in SSO.  iOS 9 Update:  Things still seem to function largely the same in iOS 9 in my (admittedly limited) testing.

In case you missed it, Apple’s recent iOS 7 release added a bunch of new business/enterprise-friendly features. One of the fine gentlemen at MobileIron wrote a nice summary covering several of them. While per-app-VPN and some of the new MDM capabilities look pretty cool, the one I’m most interested in is the “Enterprise SSO” feature.

If you’ve ever tried to use an iOS device in an enterprise environment that takes Windows Authentication for granted, you’ve probably quickly grown frustrated at the shear number of times you see the lovely dialog box in Safari asking you for a username and password. That’s not really the worst of it, because at least Safari is smart enough to prompt you, where some apps just silently fail without ever giving you the option to authenticate. The SSO feature is intended to solve this.

Unsurprisingly, Apple has chosen to use Kerberos as the core of their SSO support iOS, just like they do in it’s older brother – Mac OS. I’m not going to cover how to make your web application support SPNEGO and Kerberos in this post, but it’s really not as hard as many people make it out to be, particularly if you are a Windows/Active Directory shop. Let’s look at how you can take advantage of it assuming you have it working.

The key to “enterprisifying” your iOS devices with things like app restrictions, VPN settings, and PKI certificates revolves around configuration profiles and SSO is no different. Unfortunately, at the time I’m writing this Apple hasn’t yet updated the iPhone Configuration Utility to know how to generate profiles for the new iOS 7 feature. So, we get to do this the hard way – hand-creating an XML file with the proper config profile syntax. Unless you are some kind of blog-reading robot for whom XML syntax is a native language, this might not be the easiest task in the world, so I’m going to help you out with the contents for a “sample” profile at the bottom of the post. You can just grab this text and paste it into a plain text file with a .mobileconfig extension (e.g. kerberos.mobileconfig).

There are a few parts you’ll want to pay attention to, and then hopefully you should be off and running.

  1. As with any example you grab from the web, unless you are referencing a “well-known” identifier for some kind of resources, you’ll want to make your own GUIDs and replace the ones you see in the example.
  2. Anywhere you see “com.mycompany”, you’ll probably want to use your own domain
  3. In the string value for the “Realm” key, you’ll want to put your own Kerberos realm. In the case of Active Directory, that’s usually going to be an AD domain
  4. The URLPrefixMatches section is key, because iOS is only going to try to get and use the Kerberos ticket for URLs that match one of these. Unfortunately there is currently no “suffix” option, so you can’t just do http:⁄ ⁄ *.mycompany.com. As the name implies, you can use a prefix, but doing a “blanket” prefix like http:⁄ ⁄ * seems like it would be a pretty bad idea.
  5. Eventually I assume there will be other apps that support Kerberos, and then you might list their bundle ids in the AppIdentifierMatches section, but for now using Safari is an interesting place to start
  6. Save the modified file and then deploy it to your device. Unless you also happen to be an MDM administrator who knows how to use your MDM product to deploy these files, the easiest way to do this is probably just to email yourself the .mobileconfig file and then open it is as an attachment on the device. You’ll get a couple of security warnings this way, but it works. When the profile is geting installed you should also be prompted to enter a “principal name” for the specified realm. Generally speaking this will be your login id/username.

Now that you have a SSO configuration profile on your iOS 7 device, you can try it out. If you open up Safari and go to one of the sites that matches an entry in your URLPrefixMatches section, you’ll see a login dialog that looks a bit different than the typical “popup” prompt you’d see for a site requesting Basic or Negotiate-based authentication. Once you’ve entered your password, it should let you into the site. The beautiful part is that when you go to the next site in your list of URLs, you won’t get prompted at all, and you should be able to go revisit these sites in Safari over and over (up to the ticket expiration time for your Kerberos environment) without re-entering your password. Single sign-on is a beautiful thing!
KerbLogin

Don’t get discouraged if it doesn’t work right away. The diagnostic logs from your device can be very helpful in figuring out what’s going on. I know they pointed me in the right direction for fixing up the .mobileconfig file several times. I’d be remiss if I didn’t tip my hat to Craig Newell, a mobile tech guru at VMWare for being kind enough to post some similar sample info in the Apple Developer forums that was super helpful in getting this working.





	PayloadContent
	
        
            PayloadType
            com.apple.sso
            PayloadVersion
            1
            PayloadIdentifier
            com.mycompany.sso.test.kerberos
            PayloadUUID
            132013d0-faff-11e2-b778-0800200c9a66
            PayloadDisplayName
            SSO profile for my enterprise
            PayloadDescription
            Configures Kerberos Single Sign On.
            PayloadOrganization
            
            Name
            Kerberos Config
            Kerberos
            
                Realm
                AD-DOMAIN.MYCOMPANY.COM
                URLPrefixMatches
                
                    https://kerbenabledapp.mycompany.com
                    http://nonstandardapp.mycompany.com:16089
		    http://commonhostnameprefix*
                
                AppIdentifierMatches
                
                    com.apple.mobilesafari
                
            
        
	
	PayloadDescription
	Sets up Safari to use Kerberos SSO for certain URLs
	PayloadDisplayName
	KerberosConfigProfile
	PayloadIdentifier
	com.mycompany.ssoconfig
	PayloadOrganization
	MyCompany
	PayloadRemovalDisallowed
	
	PayloadType
	Configuration
	PayloadUUID
	B55B5378-6709-40EB-A380-76F59C3B7D86
	PayloadVersion
	1


FAST Search for a specific External Content Type

I hadn’t really had a chance to do much with FAST Search, beyond your standard demo-ware installs and use cases, but I’ve been anxious to have an excuse to dig into it more.

The combination of beefed up BCS/BDC support in SharePoint 2010 plus FAST search seems like it would be a pretty powerful combination, especially if you have fairly large sets of data in your LOB applications and you don’t want to hammer them with constant, open-ended queries.  In this case I was looking to be able to limit a search to return results of only a specific External Content Type (ECT).  In the past, with SharePoint Search I’ve been able to just use the ContentType managed property for this (there’s a default mapping that brings in the ows_ContentType crawled property and maps it to this).  However, it seems that for external content types, this property does not get set.

One option here would be to just setup a unique content source for each type of BCS entity that you want to bring in, and use the content source as part of the query.  That seems pretty ridiculous for scenarios where you either have: a) a lot of entities/ECTs or entities that are related to one another.  Fortunately, poking through the available crawled properties in my FAST Query SSA I found that there is one called “entityname” under the “Business Data” category.   Mapping this to the ContentType managed property worked like a charm.  I can now go ahead and send a query like the one included below to the SharePoint Search Web service and get back only results that are of the type that I care about. BTW, many thanks to the author of the FAST for SharePoint Query Tool. It’s the FAST-specific version of the SharePoint Search Tool I mentioned really liking in a previous post.

<QueryPacket Revision="1000">
  <Query>
    <Context>
      <QueryText language="en-US" type="FQL">filter(contenttype:equals("MyBCSEntity"))</QueryText>
    </Context>
    <SupportedFormats Format="urn:Microsoft.Search.Response.Document.Document" />
    <ResultProvider>FASTSearch</ResultProvider>
    <Range>
      <StartAt>1</StartAt>
      <Count>20</Count>
    </Range>
    <EnableStemming>true</EnableStemming>
    <IncludeRelevantResults>true</IncludeRelevantResults>
    <ImplicitAndBehavior>true</ImplicitAndBehavior>
    <TrimDuplicates>true</TrimDuplicates>
    <Properties>
      <Property name="Rank" />
      <Property name="Title" />
      <Property name="Author" />
      <Property name="Size" />
      <Property name="Path" />
    </Properties>
  </Query>
</QueryPacket>

Download SharePoint User Profile Photos Using PowerShell and Search Web Services

Have you ever wanted to grab all of the user profile photos in your SharePoint environment, whether it was to make sure none of them were inappropriate or just to have a laugh at some of the creativity?  It can be pretty entertaining, even if you don’t have an HR-ish reason for wanting to check on them.

I was recently asked to put together some way for a small group of users to easily look through all the pictures that had been uploaded.   The idea was for this to work from a client machine with no server-side installs would be needed.  This meantusing the object model was out (no Client OM in MOSS remember).  Fortunately in this environment all the user profiles are indexed to enable the “People search” capability, so the search web service is a handy and performant way to grab a few profile attributes for all the users who have submitted a profile.

I started by figuring out the syntax of the query I wanted to use to get the results back.  If you don’t already use it, I can’t recommend the SharePoint Search Service Tool enough.  The standalone executable is a free download from Codeplex that some really sharp people from Microsoft were kind enough to share, and it’s a very educational and useful tool if you want to dig into the SharePoint search engines native language a bit.  Generally I find I can just pick the scope and first few attributes I want to include in the results and then tweak the query syntax by hand.  One of the most powerful aspects is that the search tool lets you use SQL Syntax for the queries, which opens up a whole bunch of possibilities that the OOTB search web parts can’t do.  In the case, the crux of the query was to use the “People” scope, pick the attributes I wanted”, and make sure I only grabbed profiles that actually had a PictureURL populated.  You might think you’d be able to do something like PictureURL is not null, but despite being legitimate syntax it doesn’t appear to actually work.  For my purposes, I just ended up a “like (h%)”, which got the job down even though it may not be the most efficient predicate ever.

At this point, I had a nice chunk of XML for the QueryPacket to send to the search web service, as well as a pretty good idea of what the XML in the response was going to look like.  I don’t claim to be an XSLT master, but it’s not rocket science to turn the XML response into a really simple HTML page that shows the thumbnail versions of all the images, especially if you have a nice XML/XSL editor at your disposal to help with debugging.  Using the auto-generated thumbnails is highly recommended, by the way, unless you like seeing your browser sweat bullets trying to layout thousands of 100K-5MB images on a single page.  Like I mentioned before, it’s pretty funny what people will upload even though it all ends up in a 150×150 square.

I now had a one-off version of the desired result, but didn’t want to want to burden the actual customers with needing to know how to use things like the Search Service Tool, even as much as I like it.  I needed something quick and easy to automate this process, and for most SharePoint folks these days “quick and easy automation = PowerShell”.  After saving the QueryPacket XML and .xsl file for transforming the response, I just need a handy way to call the search web service from PowerShell.  At first I used the “old school” method of just generating and compiling a proxy stub with wsdl.exe and calling that from PowerShell.  After a bit more search engine fun, I found that there is a New-WebServiceProxy cmdlet built into Powershell to handle this for you.

Now I have 3 easy-to-distribute text files, and as long as the user can run the PowerShell installer, which is pretty straightforward, they can just unzip the files, right-click the .ps1 to choose “Run with PowerShell”, and enjoy perusing a large HTML file with a bunch of images that link back to the user profile page.  The actual HTML could be made much fancier, and obviously you may care about a different set of profile properties than I’m pulling.

This was a fun little exercise that actually served a practical purpose in this case, but might otherwise be a nice, fun page to show a client for whom you’ve deployed SharePoint My Sites.  You can download and play with the source files however you wish.  I’d love to hear feedback, or other ways to extend this idea in the comments.

Off to a new frontier

Well, I’m going to be moving out of a fairly length tenure at Amgen to a consulting gig at Slalom Consulting.  I’m really excited about it, and one of the reasons is that I think I’ll have more leeway to blog about some of the interesting tech stuff we’re working on or that I’m personally looking into or dealing with.  Most of it should be right up the “MS Developer” alley.

The company seems to have a really amazing roster of consultants all over the place, and is truly kicking some tail in the Microsoft tech space.  It should be a blast.

SharePoint 2010 Class Notes – Day 3

Large list throttling is definitely a good thing, but will take some getting used to.  Steve Peschka has a nice writeup on some of the implications of list throttling.

Ted gave one of the better explanations of LINQ I’ve heard or seen, which I didn’t expect to get in a SharePoint class necessarily.  It’s welcome though because the LINQ support in SP2010 is pretty handy.

MSDN has an article on how to see what CAML gets generated by the LINQ to SharePoint provider.

SharePoint 2010 Class Notes – Day 2

We’re talking about sandbox solutions in SharePoint 2010 today.  This seems like it will be a very popular capability amongst the power user audience, but could be tedious to manage a bunch of site collections and use the same set of sandbox solutions.

An interesting solution, um, solution might be to have to establish a central repository of sandbox solution WSP files, then have some kind of Silverlight app that would let you register a series of site collections you manage, and then deploy a set of selected solutions to all of them.   Think of it as a way to manage “my” slice of the farm, without using farm-wide solutions.

Solution validators are an interesting concept.  It seems like two very useful validators would be a simple whitelist or a simple blacklist, perhaps that just read a list of solution identifiers from a SharePoint list or simple database table (if you want to avoid a circular dependency) somewhere.

Looking at visual upgrade, there are some weird issues if you want to keep running with the old UI/master pages.  They’ve stapled a new feature called Wiki Page Home Page to the team site definition to use the new Wiki Page library, which sounds like a cool idea on the surface.  However, if you have a site that uses the old UI, the new wiki pages are pretty much totally unusable because the old editor doesn’t know what to do with all the new rich editor content.  It appears though, that if you just delete the “Site Pages” library on a team site, it will revert back to the old behavior and just use the default.aspx page that has plain old web-part zones on it.   It would probably be cleaner to just deactivate the feature though.

Some VS2010 “magics”:

  • prop[tab][tab] – Create a new property
  • object.Event+=[tab][tab] – Stub out an event handler
  • override [methodname]

RPC – Form Submission Cannot Be Processed Error

We recently had a fun little issue where a seemingly random set of users with varying combinations of Windows 2000, XP, and Vista as well as Office 2003 and Office 2007 started having issues editing documents from SharePoint documents libraries.

This didn’t effect all our users, which made it trickier to debug, but when we were finally able to do a Fiddler trace it appeared that the calls failing were all going to the /_vti_bin/author.dll library.  This is a long-standing holdover from the bad-old days of FrontPage and FrontPage extensions that still gets used in several scenarios by Office clients (including SharePoint Designer).

The RPC error we kept getting was “Form submission could not be processed because it exceeded the maximum length”.  This seemed really odd since the actual request size in these traces was only a few bytes.  We knew our Maximum Upload Size value was bigger than a few bytes, having just gone through an exercise to increase it to 200MB.

I’ll just cut to the chase rather than bore you with all the dead ends we went down (Google was unfortunately not a lot of help on this one).  It turns out someone had changed our Maximum Upload Size to 2048MB.  I don’t know why this happened, but it’s slightly bigger than the maximum size noted in the stsadm docs for the max-file-post-size (2047MB max).  If you spend a lot of time with powers of 2, you’ll probably notice this is suspiciously close to one of the magic numbers that tends to result in things like overflow errors (i.e. it’s right at the maximum size of a signed 32bit integer).  My guess is that the old FrontPage RPC stuff still uses 32-bit integers to store this kind of thing, and it probably “rolled over” to look like a negative number.  So any calls to author.dll were “too large” no matter how big they actually were.  I don’t know for sure, but it seems like a high probability.

Anyhow, just change the Max Upload Size back to a more reasonable value resolved the issue and our Office clients  were happily RPC-ing away again.

Highlights from SharePoint 2010 Hands-on

I thought I’d do a quick post to capture some of my favorite (and not-so-favorite) findings from spending some time with SharePoint 2010 in the hands-on-lab room.  I’ll update this as I get time.  Perhaps it will even turn into a nice “Top 10” list or something.

Highlights

  1. You can now add field/column validation right from the UI using the same Excel-style formulas previously used for filters.  There are limits (i.e. you only get one  “error message” for the whole form instead of for each column), but this will be sufficient for many of the simple cases
  2. You can seamlessly replace the edit/new form for a list with an InfoPath version by launch InfoPath from right in the ribbon for the list.  It’s really slick and will work around a lot of those pesky “Why can’t I lay my columns out this way and label/describe them better” requests.
  3. The new list view web parts are *crazy* flexible and nice and AJAX-y.  You can also embed them all over the place (e.g. inside of a wiki content section – more on this later)
  4. The rich text editor combined with the ribbon UI is absolutely brilliant.  This blows away the editing interface for any other similar tool I’ve seen, and the promise is that it will work fine across browsers too.  Haven’t had a chance to test that, but I doubt if they would claim it if there wasn’t a lot of work going into it.  Our users are going to love, love, love this editor.
  5. Tagging is *everywhere* and extremely powerful.  This feels really well thought through, and in retrospect I’m glad we avoided trying to graft on a third party product.
  6. Document IDs – yup, you can finally have unique document ID that will never change and link to it from anywhere.

Lowlights

  1. List to Excel integration is still via the same crufty .iqy file mechanism that has been around for several SharePoint iterations now.  Not only will this mean years of the annoying/scary error dialog when opening a list in Excel, but it likely means no way to have a published Excel sheet (in Excel Services) that points back to list data.  C’mon Microsoft, is it sooooo hard to make Excel use the same integration mechanism that Access does (i.e. web services)

Jeff Teper Keynote and live demos – SPC 09

Jeff is a corporate VP in charge of SharePoint who has apparently been around it for a very long time.  I don’t really remember him from previous events for SharePoint, but I’ll take his word for it.

He put up a Bill Gates “1 slide vision” document from 1999, and it’s pretty funny how well it still applies to the challenges around integrating all the stuff SharePoint deals with together.  The man definitely had a vision.

Ugh, so far this talk is like the overly-long Academy Awards thank you list.  I’m sure everyone feels much better about themselves.

We’re going through the new version of the SharePoint “cheese wheel” or pie.  It’s now all blue, and has some tweaked titles, but largely still very similar.  I’m sure will be seeing lots of this.

Communities is one of the new concepts in the wheel.  It’s basically lumping together some of the collaboration stuff they already had (blogs, wikis, team sites, etc) with more of the social networking capabilities (tagging, “folksonomies”, meta-data management, expert identification).  The My Site experience and the profile has been revamped considerably, including a Silverlight org-chart that may be the Outlook Address Book killer.  We’ll be needing to take a close look at this at work.

ECM Scenarios.   They are taking the scalability to whole new level.  Millions of documents in a folder or list, 10’s of millions in a library, federated scenarios to make a seamless experience of 100s of millions across several libraries.  This will definitely require some in-depth investigation later, but it certainly sounds promising.  They are going for the jugular vs Documentum/Livelink here.  Second big applause item here – content types can be managed at any level, including across farms.  They can store large objects like videos for digital asset management scenarios (goodbye Podcast Kit for SharePoint).  “Document sets” are a new feature that is very much like virtual documents in Documentum.  They’ve ramped up the governance, records management, compliance facilities.

Search is changed pretty significantly as you can now have either “standard” search or FAST search plugged in.  These are supposed to have a consistent programming model, but he did not say if it is the same as the existing model.  My guess is no.  They are trying to leverage the fact that they know our organizational structure to increase search relevancy.  I’ve been hoping for this for years, as they have significant competitive advantage there vs other search platforms given the OS/Office footprint.  Also doing things like automatically figuring out name “typos” in searching for people based on the way the names might be mispronounced phonetically.

On to BI.  They are pushing the whole PerformancePoint suite into SharePoint as a throw-in now.  Excel services now looks like a much more complete product.  Things like charting web parts, sparklines, etc are now built-in.  On a very geeky note, Excel calculation services are now available as REST feeds, so you can link to specific Excel data from anywhere on the web.  Sweet!  They’re going to have some ways to bind  Visio charts to data and then expose this in SharePoint.  That could either be useless or tremendous, depending on the implementation.

They are finally unveiling “Project Gemni” as SQL Server PowerPivot.  This is something that can either be run on the client (in an Excel flavor) or the server (a SharePoint flavor) to slice and dice millions of rows very rapidly.

Admin Notes

Big picture, their SharePoint Online offering made them highly motivated to have a scalable way to manage large SharePoint environments, so there have been a lot of improvements here.  They finally put the Admin Web UI through their usability labs.  If you’ve ever seen the Central Admin page in WSS 3, you’d know this was a long overdue idea.

PowerShell is fully baked into this release for the command-line administration of SharePoint.  I don’t think this means the death of stsadm.  It’s more of a complement.  Also you can manage the farm remotely from a client machine (Windows 7).  Yes!

Centralized logging database for SharePoint environment that will have a publicized structure so we can use whatever we like to report against it.

Upgrade experience should be much more seamless than 2003-2007.  As they already demoed in preview videos,

Demos

Someone named Jared up to do demos.  No, not the Subway guy.

He’s showing the ribbon UI and the rich text editing.  Wow, he just popped into “wiki syntax” to point to an arbitrary piece of content in the site.  It’s hard to describe but, typing the [[ brought up a way to pick any other available pages, kind of like Intellisense in Visual Studio.  It’s really slick!

Showing the new ribbon UI also includes the ability to perform operations like “Checkin” on multiple documents at once.  This is one of those things that seems like it always should have worked that way, but it finally does now.

Very slick integration between the Office Clients and SharePoint.  For instance, now opening a SharePoint site in the File dialog in office lets you browse by meta-data not just treat it like a bunch of folders.

You can definitely tell they’ve listened to customer feedback on pain points.  That has always been a Microsoft strengths.

On to the new MySite/social stuff.

Hmm, a tabbed interface for all the sections on your profile  page (e.g. organization, my blog posts, etc).  I think I knew someone once who liked that idea and proposed it as a customization for SharePoint 2007 My Site profiles. :)  There’s now a “work notes” section on profile pages that is a blatant rip-off of the Facebook Wall, but lots of people seem to like that.

Ratings are built-in and ubiquitous.  Nice!

They are so going to kill Google Docs and Spreadsheets with the online versions of the Office apps.  In demoing the “co-editing” features, Jared has a great line.  “The nice thing about co-editing is that it allows me to watch other people do my work for me.”   Well, when you put it that way, sign me up.  REST-ful feeds for Excel look pretty awesome too.  He just showed how to link to a live chart directly from Excel services using the “Insert Picture” capability in  a blog post.  It’s a really powerful concept.

He’s really flying through stuff here.  Showing search, which now has wildcards built-in.  People search is rerally slick with that new Silverlight Orgchart gizmo (that’s not really a new technical term.  Why not, we use gadget and widget already?).

So PowerPivot appears to scale pretty well.  He’s working on an Excel spreadsheet reporting 101 million rows.  It’s smoking fast filtering data.  Not sure what this is based on in the backend.  It might be use Analysis Services cubes or something.  Combining this with the ability to publish a workbook to a SharePoint site is extremely powerful.  Even with an investment in another enterprise BI tool (e.g. Cognos,  Business Objects) this is something that bears looking into.

It looks like there is some flavor of SharePoint Workspace/Groove available for Windows Mobile, but no demo so it may not be baked yet.

Arpan Shah handles the admin demos.  There is a new “Health Analyzer” section in Central Admin that will try to identify issues in your farm based on a rule set.  This should be pretty helpful, since there are so many places to look for this stuff now.

He’s also showing the ability to set thresholds on large lists to allow lots of items in a list, but put governors in place to keep inefficient use of them from overwhelming your environment.  Not that this could ever happen in SharePoint 2007 or anything. 😉