Fabian Williams SharePoint Blog

Solving problems with SharePoint day and night

Wrap-up on my SPSDC Workflow Session

Thanks

Another year and another successful even by the Organizers (Dan, Adam, Rachel) and all the Volunteers at SharePoint Saturday Washington D.C. – Reston.  I was honored to be an accepted speaker and I delivered this session “The Evolution of Workflows in SharePoint 2013 – A Story about External Calls with REST & JSON”.  This was truly a magnificent event with a great turn out, good mix of attendees, and they were very engaged, asked serious and deliberate questions , and didn’t even mind as yet again i go a little bit over in my Live Demos.

Appreciation

Its always great to meet face to face the folks that you interact with daily on social media in our SharePoint community, foster new relationships, maintain old ones, and Share a Pint at the end of a day where we all walk away with a bit more knowledge in this technology we all share. I was very please for all the attendees that made my session ‘standing room only’ and if any of you all have pics, please email them to me, i would really appreciate them for my blog post. 

My Slide deck may be picked up here

V__C0D1

What’s Next

Make sure to attend the inaugural #SharePoint Saturday, the one that started it all SharePoint Saturday Virginia Beach on January 11th 2014 & of course stop by my session (well after you register here) on Optimizing your SharePoint 2013 with BCS an External Data where we will discuss strategies, security and all the various models to interact with your LOB systems.

As well, I will be delivering three (3) Sessions at SharePoint Conference 2014 March 3rd through 6th 2014 (#SPC14 or @SPConf) in Vegas check out this link for my sessions.

I hope to see you there.

See oonuh latah

December 9, 2013 Posted by | JSON, Public Speaking, REST, SharePoint Designer 2013, Where is Fabian, Workflows | Leave a comment

Why it pays to have Smart Friends to Encourage you

Why write this blog post?

Sure its late and YES I could be rotting my brain away on the Idiot Box, but what started off as a Taunt by Ram turned out to consume my life for 2 days straight but with useful results, challenges that were overcome, and a better appreciation for not just Cloud, but the integration of Cloud and On-Prem using Windows Azure.  Let me tell you what I did, and how easy it was… in the end, plus the real implications of it.

First a little history – Pre Taunt

So, as I prepare for my sessions at #SPLive in Orlando in 3 weeks, I wanted to knock out a few Demos and make sure that I could showcase just about anything anyone asked of me based on the Session Abstract/Topic. In furtherance of that, I decided to do the following, build out solutions employing

SharePoint Designer 2013

  • External Content Type using a Native SQL Data Connection
  • External Content Type using an On-Prem (i.e. Hosted on one of my Dev Rig VMs) Windows Communications Foundation [WCF] Data Connection – JSON enabled
  • External Content Type using a .NET Assembly (built on my Visual Studio 2012, Deployed as a Farm Solution to my On-Prem SP2013) Data Connection
  • Workflow using the same On-Prem WCF above in my On-Prem Farm

Visual Studio 2012

  • A Locally IIS hosted WCF Service Library that exposes Northwind data from a Local SQL Server 2012 Box
  • A Locally IIS hosted OData Service Endpoint that exposes Northwind data from a Local SQL Server 2012 Box
  • External Content Type built as a .NET Assembly to be deployed to my ON-Prem SharePoint 2013 Farm
  • External Content Type AND External List SharePoint 2013 App built as an OData Sourced Application deployed to my ON-Prem SharePoint 2013 Farm

For every Instance of the WCF and the App Model External Content Type, I also targeted my Office 365 Tennant to surface the Data as well. I was pretty satisfied with myself until my good friend sent this tweet, and I know it was in jest…

image

but… I took it as a challenge and, I rarely back down from a challenge 🙂 so, since I have a MSDN subscription, there was “really” nothing holding me back besides my inexperience creating Azure Web Roles, which is what you need to do in order to have the requisite Data Access Points (URI) to create External Content Types and potentially Workflows. 

What Next you say…

So, with a little research on Google scratch that, I mean Bing, I found a few MSDN, and TechNet blogs on how to create Web Roles, although not many of them “accurately” showed you how to use Visual Studio to DEPLOY your solution back to Azure. Anyway, it was surprisingly easy, although initially it was somewhat intimidating and i did get snagged on a GOTCHA which i will blog about later when doing a ASP.NET Web Role where it requires you to lower your version of the System.Data.Client assembly from 5.0 to 4.something in order for it work in Azure, i say that because it worked perfectly when I debugged it locally on my Visual Studio IIS. I also had to create an Azure SQL Instance and used a script I downloaded from CodePlex to restore a Northwind Database to my Azure SQL Instance and then I set up firewall rules to allow me to access it over the internet.

But with quick turn around and because i could just Refactor my original On-Prem code logic, I was able to create the following

Visual Studio 2012

  • Azure ASP.NET Web Role that published as a Cloud Service which used OData via ADO.NET Entity Framework to get a Cloud based URI that i can do CRUD operations on
  • Azure WCF Web Role that published as a Cloud Service which used WCF via ADO.NET Entity Framework to get a Cloud based URI that i can do CRUD operations on as well

    image

Now I can really Mix and Match, I can

  1. Use Azure (Cloud) Hosted Data via an ON-Prem WCF or OData End Point and surface that information both on my ON-Prem SharePoint or my Office 365 Instance
  2. Use Azure (Cloud) Hosted Data via an Azure Cloud Service WCF or OData End Point and surface that information both on my ON-Prem SharePoint or my Office 365 Instance
  3. Plus everything I had when I started off

The takeaway here is that with very little investments in time and effort I extended my solution to a “MODERN” approach. I had Cloud in the sentence 🙂

 

Other Positive Implications

So, what made this so easy for me was I signed up for TFS Online here http://tfs.visualstudio.com/en-us/tfs-welcome.aspx and EVEN WITHOUT a MSDN Subscription you can sign up for FREE and get 5 accounts in one instance.

image

What I think is GOLD is…

  • You have a place to store all your CODE/ Work and before you ask, it supports different formats (see image below)
    image
  • Your Code/Work is accessible from Anywhere you have an Internet Connection now
  • You can reliably share your Code/Work with anyone now, rather than Zipping it up and Email it
  • Need Help with some of you work… Invite people as smart or smarter than you to review our code, many hands make light load 🙂
  • Most beneficial of all is that you can configure Continual Integration. I know my good friend Jeremy Thake always talk about Application Lifecycle Management (ALM) and you can employ some of those techniques from Azure through TFS Online and your Visual Studio with multiple Team Member… and guess what… FOR FREE
    image

So that’s all I have to say, I guess we really Paid It Forward RAM, you encouraged me to do this, I in turn, documented my efforts and hope that someone else can take it from here.

 

Cheers all, have a great night. Oh yeah, VS 2013 is out now. GO get it.

October 17, 2013 Posted by | Azure, JSON, OData, REST, SharePoint 2013, SharePoint 2013 Workflows, SharePoint Designer 2013, SharePoint Development, SQL Server, Visual Studio 2012, WCF | , , | Leave a comment

ACTUALLY RESOLVED: Unable to Create List using SharePoint 2013 REST API in SPD2013

Are you ready for this?

Im never satisfied with the answer “No you cant do that”, not as a Developer 🙂 So, i began to read through some TechNet and MSDN articles for Workflow and found this Gem. http://msdn.microsoft.com/en-us/library/jj822159.aspx 

What this Gem does is EXACTLY what Chris Givens found out and did in his PowerShell but as we discussed in my “Resolved post”, we cant run those scripts inside Office 365 SharePoint Online. So, it seems that you have to activate a Site Feature in your Site that you need this “elevated permission” then you need to copy the App Principal ID from the one you want elevated then take that to a “hidden” link inside the App Catalog Site http://{hostname}/{catalog site}/_layouts/15/appinv.aspx and then work your magic to elevate the permissions via an XML node copy / paste magic.

Anyway the GOOD NEWS is that I have tested this in SharePoint Online, and it works like a champ, see below for screenshots.

If you would like the full history of this, see the following post in chronology

  1. https://fabiangwilliams.wordpress.com/2013/09/06/help-unable-to-create-list-using-sharepoint-2013-rest-api-in-spd2013/
  2. https://fabiangwilliams.wordpress.com/2013/09/07/resolved-unable-to-create-list-using-sharepoint-2013-rest-api-in-spd2013/

Proving it out

First me activating the Feature

image

Next grabbing the App Principal ID needed for the Workflow in question. Now i purposely cut off all of the ID, i dont want some crafty person out there using it. Just realize that this is the same account that Chris and I talked about in the PowerShell

image

then moving the piece of Workflow logic that does the List Creation inside the App Step inside SharePoint Designer

image

and finally re-running the Workflow will yield the below in the Workflow History

image

and just to bring it home, here is the new List or (App i guess) Created

image

 

Cheers FGW

September 8, 2013 Posted by | Azure, JSON, Office 365, SharePoint 2013, SharePoint 2013 Workflows, SharePoint Designer 2013, SharePoint How-To, Strange Stuff | 13 Comments

RESOLVED: Unable to Create List using SharePoint 2013 REST API in SPD2013

Summation

Ill start this post out by saying that this is why i so LOVE the SharePoint Community. It is a true example of "many hands make light work”, first Jim Bob Howard @jbhoward one of my @PlanetTech work colleagues faced a problem, since it was a topic of interest to me, I decided to put some time into it as well. I was able to prove out most of the issues, but there was this last one (see blog post to what this is referring to in a link below) that just didnt make sense.  What do i do when that happens, well I’ve never been ashamed to reach out to my friends in the SharePoint Community so I blogged the issue, showed what we did so far, and without tagging anyone in the tweet we had folks from the UK joining in the conversation, folks from all Time Zones here in the United States chiming in as well. Late Friday night I emailed a copy of the Workflow to my good buddy @givenscj Chris Givens.  I really wasn’t surprised when even on a Saturday, Chris called me up on the phone, and we spent like 2 hours on a Screen Share looking at ULS logs, then he went off on his own, and within an hour, he came back with a solution. 

The original problem was posted on the forums here

My Plea for help on my blog which gives more details than the forum post allows is here. This post is basically closing the loop on the problem

Findings

Chris blogged his findings here, go look at it, but the main point ill just borrow from is blog…

“…The main issue of Fabian’s was that when he made the Web Service call he would pass a pre-created FedAuth token (yeah I told him he would have to figure out how to generate this in the future) which established a context for him.  This context was as his SharePoint admin account, but the actual running account (the "actor") was the Workflow service.  Of course the SP Admin account can do whatever it wants, but the workflow service cannot.  This workflow service account is NOT the account that the actual workflow manager backend window service is running under.”

Screen Shots form what Chris and I worked on

First we wanted to see what was happening ON Prem with this both from a Fiddler standpoint as well as SharePoint Designer 2013, here is what we saw. The offending item was the account it is running under MS.SP.EXT which is the App Principal for the Workflow from Workflow Manager. Further down you will see this inside the Content Database and the permission it is set to run under.

tmwwgjzs

with a little more detail below

e5mofjsq

Below is a peek into the App Principal Table. This table as Chris describes is where the App Principal Accounts live, you can see that i have a few that are Titled as my Workflow Apps and one called just “Workflow” and it is that one if you look under “Name” has MS.SP.EXT 🙂

image

Now, lets go look at the permissions, now mind you, if you look at permission it is set to “5” right now for the ID 1, that was set to “3” before i ran the PowerShell that Chris has on his web site and i will put below for complexness.

 image

Here is the PowerShell below

PS C:\Users\spadmin> $web = Get-SPWeb http://farm1sp13svr1
PS C:\Users\spadmin> $appworkflow = Get-SPAppPrincipal -site http://farm1sp13svr1 -nameidentifier 31ed5e6e-979b-redacted
PS C:\Users\spadmin> Set-SPAppPrincipalPermission -AppPrincipal $appworkflow -site $web -right fullcontrol -scope site
PS C:\Users\spadmin>

Once I did that, all we did next was re-Run the workflow, NO other changes, here was our result

image

and in All Site Contents

image

 

 

Finally

So, all is well in the world of SharePoint again. I hope Chris goes and puts his solution as the answer to the question in the @Office365 Forums so we can have a record out there as well. I hope this helps someone else that falls prey to this, I agree with Chris, this is the first time I have ever seen anyone blog or talk about this in a public setting.

Cheers

Fabian Williams with contributions from Chris Givens

UPDATE SECTION

So, because i wanted this to work in Office 365 as well, we needed to find out if these PowerShell options are available there too. For completeness I loaded up the ISE and I have here all the options that you can do with the App Model for GETS and SETS, both for On Prem and OnLine, you can see that the Online version is very limited

ON PREM

clip_image001clip_image002

 

and sad to say I could get EVERYTHING from ONLINE in one screenshot. 😦 sad panda

image

If you want to know more about Powershell for SharePoint Online regarding how to set it up in your environment, see Patrick @pcfromdc Currans blog post here.

 

END UPDATE SECTION

 

September 7, 2013 Posted by | Azure, JSON, Office 365, REST, SharePoint 2013, SharePoint 2013 Workflows, SharePoint Designer 2013, SharePoint Error/Resolution, SharePoint How-To, SharePoint Online | 1 Comment

HELP: Unable to Create List using SharePoint 2013 REST API in SPD2013

Précis

So, I am took up @Office365 on their offer of help, but then they directed me to the forums because they said the limitation of twitter made it difficult to respond to my tweet for help

image

So i set about doing that this morning, however even the forums have limits when you try to paint a thorough picture of your problem using screen shots due to image sizes. see here

image

So, i will just blog it and point back to this blog post using the Forum as the initial pointer for anyone who wants to help, fortunately WordPress give me more freedom in images and i can be verbose. So here goes…

Problem Stated

The objective is to Create a new List in SharePoint Online Wave 15 via REST https://site/_api/web/lists and then passing the necessary headers. Also, if I do a GET using the same headers, IT WORKS, this only happens on POST with the addition payload of Request Content(Body) that I am sending, but in Fiddler "using" the same payload, I can create my list. I’ve blogged about this topic and how I go about doing REST via SPD and Fiddler here and here and here, the first and last are probably most appropriate for this line of question. Below are some screen shots of what I am experiencing now. I thank you in advance for any assistance you can provide.

Below are the Request Headers

1

Below is me constructing what will be the Request Content (Body) but to get the desired feel, I had to put the __metadata property inside its own dictionary object to get the curly braces and key value pair, you will see below

2

Below you will see me wrapping this variable inside another to get {‘__metadata’:{‘type’:’SP.List’} to come out properly

3

Next you will see my call to HTTP web Service properties

 4

Next you will see my results. this is my parsing the JSON returned in the Response Code and Response Body

 5

Proof that the same thing works in Fiddler

Here is me doing the same thing in Fiddler. and i also updated my Digest so that it would be current because i know it expires after a while

 

image 

and

image

and the list in O365 is created here

image

 

Update Section

This section will be filled with things I have tried either on my own or on suggestions.

The first thing i will do is take Hugh @hughajwood Wood suggestions and use a simple example on a post I actually praised earlier and do just a simple Add of a List Item in Office 365, i will dubb it TryAlpha

Try Alpha – 09/06/2013 1212 HRS

So the first thing I did was to make the headers as the post describe, to keep things simple i used the exact nomenclature

image

now wrap that header into a Dictionary header named __metadata and add a Title field

image

next use those headers to make my REST call

image

and we can safely say that using the REST API for List Items work. Let me also make some qualifiers

  1. Before doing this, I updated my X-RequestDigest to get a new value and I updated my requestHeaders variable, that was the only thing i did

image

End TestAlpha

Begin Test Bravo

So, I got some information from Paul Schaeflein @paulschaeflein He suggested that rather than hardcoding the Digest Information [which I know also expires after a while, and that is why whenever I run the WF, I redo the contextinfo URI and get a new one] I should do a POST insider my Workflow to get the Digest Information and THEN extract the X-RequestHeader value inside a variable and use it. The thought being, it will be more current than hard coding it, and Pauls Point was that it is safer from a security standpoint if SPD was preventing ‘malicious’ usages or hardcoded values.

Well In the end, it didnt work either. here is the story

Big Picture

image

In more detail I created one header to get the ContextInfo Information i.e. No Digest information was used in that one, and then I set the value coming from the JSON WebContextInformation object mapping to the Digest into a Variable called str_FormDigest.  I then used that Variable in another Header that I have specifically for the POST that I will use to create the List Item. See below

image

And for good measure I logged the output of the Digest so i know it works. here is the Digest Info and another failed attempt

image

End TestBravo

 

 

 

 

 

 

Summary

Any help is appreciated, I will ask that you post your feedback in the comments on this blog and i will update the Ticket in the Forum pointing back to this blog post.

 

Here is the Forum Thread: http://social.msdn.microsoft.com/Forums/en-US/dd9932f5-1e3f-416f-8f0b-e8ab2b792ba1/unable-to-use-spd-2013-and-call-http-service-to-create-a-new-list-in-spo-via-rest

 

 

del.icio.us Tags:

Thanks in advance

September 6, 2013 Posted by | JSON, Office 365, REST, SharePoint 2013, SharePoint Designer 2013, SQL Server | 8 Comments

Using the Call HTTP Web Service REST Calls in SharePoint Online using SPD

Précis

So I’ve had folks say that they still cant make the leap and get my examples to work in Office 365 SharePoint Online, so this post is to Prove out that IT can be done, there is no smoke and mirrors, I just wanted folks to take what I have and use that to go to the next level. This is not like a live session where I have an interactive audience, so I am putting the info out there in his blog.

This post builds on a few out there that I have done explaining how to use SharePoint Designer 2013 with the Dictionary Object to data in JSON Format to be used inside Workflows. Now, I do admit that there are VERY FEW examples out there, if any, that show you, how to do what I am about to tackle in Office 365 SharePoint Online. So here goes.

 

Approach

As usual we have an Initialization Stage and a Workhorse Stage, in the Init stage we will set our variables and make our REST call, we will parse it and limit our dataset to what we need for our results. I think folks are getting tripped up on what is needed in the Headers, as what I am getting as feedback is that the Workflow Starts but it stalls with a message like “Retrying last request. Next attempt scheduled after…” with a correlation ID, it also gives you a Retry Now Link.

So the first thing i will show you is of course what is in my List NOW, I will show it to you via the browser and in Fiddler.

image

now here is the items in Fiddler, i do this so you can appreciate the actions I will take later on in SPD when parsing the JSON data.

image

Now lets look at how we construct the Header, in this post, i have explained why you need certain Headers, so i wont go into it here, the only difference is that I am extrapolating what I did in Fiddler and using it in SharePoint Designer… now for obvious reasons, I am NOT showing you all the Header Information since it is my Valid Token to O365.

image

Now with that done, I have constructed a Site Workflow that will loop through the items and Log it to History, what you do with it for your Use Case can be quite elaborate, you essentially can

  • Communicate with any Office 365 SharePoint Online Tennant you want from a Single Workflow
  • Cross Communicate between an On Prem Farm and a Cloud Farm in one Workflow
  • Create List, Libraries, List Items, and do full CRUD operations

If you go and look at the REST API endpoints from that link i gave you before by Kirk Evans, you can see that its quite a bunch of things that are available now.

So. back to the story.

First Part of the Workflow

image

Second Pare of the Workflow

image

Next lets fire off this Workflow and look at the results.

Proof

Next we go to Site Contents, Site Workflows and click the one we just created after publishing

We should expect to see the below in our History List

image

and upon closer inspection of the Workflow

image

 

Summary

Now what I will say because I have worked quite a bit with SharePoint Online Wave 15 is that it can get squirrely and can crap out on you for no good apparent reason, it gives you obscure feedback when it does and often times it just says “something bad has happened..” and if you Refresh the browser, its back to normal. I don’t pretend to understand why, I don’t have eyes into the Data Centre nor access to the ULS Logs, so I am as much as in the blind as you are. BUT this is the reason why i do Fiddler first then SPD after…visibility.

I hope this helps! Cheers.

 

September 4, 2013 Posted by | JSON, Office 365, REST, SharePoint 2013 Workflows, SharePoint Designer 2013, SharePoint How-To, SharePoint Online, Workflows | , , | 4 Comments

Creating a SharePoint Timer Job using SharePoint Designer 2013

What do i really mean?

Now that I have got your attention with that eye catching title, let me expand and qualify it.  So, what I will be doing exactly is

  1. Inspecting a List (a Task List actually) for Tasks that are Due and Overdue.
  2. I will be checking them nightly (every 24 hours to be exact)
  3. If task are Over Due, then I will take ‘some’ action on them, in this instance, I will be changing a KPI field but this can be quite exhaustive based on your own particular use case. see below for one i have in mind.
  4. Pause the Workflow for 24 hours and Repeat.

By the way, this CAN also be used in SharePoint Online

Use Case

In this use case I have been charged to monitor a Task list (multiple task list from varying SharePoint Sites/ Site Collections) and aggregating them. Once complete, then we need to serve up the tasks that are overdue into a KPI dashboard and send the email link to managers of those offending taskees.

For my POC demo here I will show how to get 1 Site Collection but to get multiple all you have to do is make a separate REST call to that particular Lists API, and the rest is easy.

Pre-Requisites and Technologies Used

In this example I am using an On Premises SharePoint 2013 with Workflows enabled, I am also using SharePoint Designer 2013 as well, that is if for tooling. i.e. NO CODE.  I will be using the SharePoint 2013 REST API to read the SharePoint Task Lists as my Entry Point URI. The generic url will be /_api/web/lists/getListByTitle(‘’)/items">/_api/web/lists/getListByTitle(‘’)/items">http://<sitename>/_api/web/lists/getListByTitle(‘<listNameHere>’)/items which fetches List Items from a Named list using oData. Now by default this will be returned to your in XML, so in order to use this in SharePoint Designer which requires JSON inside the Dictionary Object (see my post here if you need a refresher on that) you will need to modify the headers to accept JSON using the Accept Headers. The rest is a matter of using Loops to iterate through the List Items returned, and pausing for the time allotted.

Approach

As usual I put my URI inside a Variable to obscure the name & any API Keys if any, this time there was no need but it just makes for better programming especially if there are a lot and I want to swap them out from time to time without affecting the core program

image

the full list of properties from the Call HTTP web service is located below since it is truncated in the image

image

The Request Headers are “Accept: application/json; odata=verbose” and you can place that into a string type as well and call it into the HTTP Web Service Properties as well inside a Workflow Dictionary Variable

image

Once you have done that, then as seen in the first image and in my previous blog post on the same topic, you parse the JSON results to get the part of the dataset that you want, in this case i want what is under the “d/results” node and to do that I use the Get Item from Dictionary and parse out what i need then i count the items for good measure, save the “Count” as a variable to be used as my upper bound in my Loop Counter and log it out.

Do Business Logic

After setting up the stage to get the URI, return the JSON data, Parse It to the Node you want, and get a Count, the next thing is to do the business logic.

What you will see below in the image is us using the Variable Count as the upperbound in our Loop and then getting the JSON data into Local Variables to be manipulated in the Workflow Logic.

//MY SOAPBOX

So, i have been talking about this for a while as most people who have known me for a long time, know be because of my work and efforts in SharePoint BCS.  But i have actually moved my thoughts in some aspects to what BCS allows (even in 2013) and what Workflows gives you interms of interacting with External Data. You see my point is, if you do not need to look at that data, i.e. External List, or need to Search on that Data i.e. External Content Type as a Content Source, I THINK that it is to your best interest from a simplicity standpoint, a performance standpoint, best of breed standpoint… to do this as a Workflow calling External Web Services and manipulate the data how you want and then get rid of it (i.e. the Variables) when done. That way the data is only used for its intended purposes then goes away.

//END SOAPBOX

Back to our story, what we do next is inspect the Due Date and use Conditional Statements to affect changes to the Fields for what you need. in this instance I am updating a KPI field based on how far along or how overdue a Due date is on a Task List. Now the Use case here is that I can do this on Any Amount of List both On PREM and in Office 365 in the same logic, I have Scale and Scope down to a Tee 🙂

 

image

 

The logic loops through each item does a Check and then updates the loop counter until it hits the “Count” variable number then exits. Since this is a Timer Job then the next thing we need to do is Pause it and wait, see below

image

Now, this workflow was designed as a Site Workflow so it can be ran independently of any Library or Lists and this guy can run forever.

Summary

So the next time someone ask you to Do a Timer Job for them, especially if they say they want it on Office 365 (SharePoint Online) and it involves a use case similar to mine or at the very least is looking at affecting changes to a List/Library, consider doing a Site Workflow with Conditional Logic and Pause Duration. Its really is that simple.

If this was being done in SharePoint Online, you would also need to capture and pass along the oAuth Tokens FedAuth and rfTa in your dictionary object.

Cheers and Happy SharePointing

September 4, 2013 Posted by | JSON, Office 365, REST, SharePoint 2013, SharePoint Designer 2013, SharePoint How-To, SharePoint Online, Workflows | , , , | 13 Comments

More on SharePoint 2013 REST API with Fiddler and SPD

Précis

So, as of lately when it comes to SharePoint 2013 Workflow, CSOM with Managed Code, REST API, I have been trying to see how far down the rabbit hole i can go, and talking about what I find.  As a result, I have been fielding quite a few questions on the topic; although there is quite a bit of information out there by Kirk (@kaevans) Evans here on MSDN and others, what i have find is that they discuss a lot of the Reading Data, usages in C# and JavaScript and they elude to debugging with Fiddler.  I on the other hand, like to test my work in Fiddler then take it into SPD (SharePoint Designer 2013) or Visual Studio 2012 for either work with Console Apps, Workflows, Event Receivers etc. It also gives me the opportunity to see what is happening under the layers of SPD, or obscured by Visual Studio Activities. Now there are times when that door will get slammed shut in you face especially around SharePoint Online Office365 and then i just rely on Wictor (@wictor )Wilens MSO Helper in a Console App to peruse around. This post will show you how to Add data into a SharePoint List (i.e. ListItem) using the REST API via Fiddler and How to Create a SharePoint LIST as i couldn’t find a decent example out there after suggesting to a work colleague that they use Fiddler to map out their work then do it in SPD. On the other hand, there is a great post here by Borislav Grgic on how to use Add an Item to a SharePoint List using the REST API and POST Method in SharePoint Designer 2013.

What Major Obstacles you will need to Overcome

This is what I found lacking when I did research on how to achieve the stated objectives above.  Most of the examples just told you to either “..type this” or “create a dictionary object to to that…”, and some just omitted major steps that you will get tripped up in and no way to circumvent it. So, taking this by the numbers, lets first assume that you want to get back some List data from SharePoint 2013, we will do this first On-Prem then In-Cloud (Office 365).

On Prem

  1. You will need to construct a Uri representing your target list such as :  //_api/lists/getListByTitle(‘’)/items">http://<site>/<web>/_api/lists/getListByTitle(‘<listNameHere>’)/items in a browser (preferably Chrome) or Fiddler
  2. You will need to set some Headers, specifically the (a) Accept and (b) Content-Type Headers to accept ‘application/json;odata=verbose’ inorder to work with the return items in JSON format which if you are using this in SPD, you MUST do in order to use the Dictionary Object, or if you want to read the data in some logical hierarchy.
  3. If you are ON Prem, and using Fiddler inside “Composer” under Options, i usually set the Automatically Authenticate Flag like so, you will see me talk about Decrypting HTTPS Traffic later on in the Cloud or HTTPS targets later on. In doing so your Cookie header will be automatically set.
    image
  4. Next ensure your Verb is set to GET and fire away.

In Da Cloud

  1. Well steps one (1) and two (2) are the same from the above, however step three (3) is a bit different.
  2. You will need to pass your Cookie Token for the Office 365 Site here, and simply put you will log into the Target Office 365 Site and using using Fiddler you set it to “Capture Traffic” after you log in. If/when you click on a List or Library as an example, inspect your Headers returned (assuming you have already gone under Tools / Fiddler Options and selected to Decrypt HTTPS Traffic) like so
    image
  3. Once you do and inspect the Headers you will find cookies for FedAuth and rtFa under the “Cookie” header. Go ahead and copy and paste that in the Request Headers section of Fiddler
  4. Next ensure your Verb is set to GET and fire away.

What if i want to POST (Add) Data to my SharePoint List?

Well, if you do, you have to do a few more things before you can actually POST that data.

  1. You will need to get the X-RequestDigest Header to send along with your POST, notice this is “not” a GET anymore. Now referencing the post again above by Kirk Evans if you look almost 3/4’s away down on the page or just Cntrl-F to find the word “contextinfo” you will find a description on how to get that Header Information in the d:FormDigestValue node.
  2. Essentially you will need to change your Fiddler VERB from a GET to a POST and send along an empty Request Body, you may need to set a “Content-Length” Request Header as well, and i usually do 104 for that length.
  3. You will get back the information that you will need to add to your Fiddler statement.
  4. In addition you will need to construct a Key/Value Pair (hence working in JSON) to pass along in the Request Content that you want to Add
  5. Now there is one more obscure piece of information that ususally trips people up that you will also need to do, and this information will form a part of your Request Content that holds your Data you are trying to add.
    1. When you did your GET for the site, there was a Node called (underscore undescore metadata) __metadata. You will need to take that value and pass it first ahead of any data you are trying to add. You will see this example below
  6. Once you do all the above, you ensure that your Verb is set to POST and fire away.

Lets look at a few Examples

So taking this from the beginning we will use a use case of a List i have called “AndyTaskListOnPrem” and we will first do some Gets, then a Post and then finally verify our work. Here is an example of the list before the exercise.

image

Next lets look at how you need to construct the GET to look at the data in JSON format in Fiddler

image

And our Outcome. Pay attention to the Metadata Node and the type = SP.Data.AndyTAskListOnPremListItem value. you will need this later on for the Adding of List Item.

image

continued

image

Next lets look at how to get the ContextData information.

Create a POST with the URI of /_api/contextinfo">http://<site>/_api/contextinfo as i have below

image

Ensure the Response Content is blank and execute that, your results should look like mine below. You will need to capture the “FormDigestValue” information for later on in your POST.

image

Next lets Create a List Item

Armed with that information we have enough to create a List Item in this SharePoint List using the
REST API and Fiddler and using the same information plug this into Dictionary Objects in SharePoint Designer along with the Make a HTTP Call to do the same, here’s how…

image

Paying attention to the POST Verb, the X-RequestDigest “Request Headers” and the __Metadata and Addition in the “Request Body” after execution the above you should get

image

a HTTP/1.1 201 Created response. and if we inspect the list we should expect to see

image

 

To round things off, lets assume our Target was Office 365 SharePoint Online, then we would expect to see our Headers contain our Auth Tokens.  I will take this to a next level by using an example someone just asked me about. I will Create a List in Office 365 “not list ITEM, an actual LIST” using the same methods described above. the KEY takeaway here is I need to Pass my Auth Tokens and ofcourse I am using a different REST API Call

Lets Create a SharePoint List using REST and Fiddler

So, in the spirit of openness, here is the REST call to get my Auth Token… well some of the token, I wasnt born yesterday 🙂

image

Here is the Context Info, key here is remember its not HTTPS and you need pass the Token as well.

image

Finally here is me creating A NEW LIST in the Web

image

and with all the confidence in the world [well this is O365 so we also cross our fingers] we expect to get a 201 Create Response back from HTTP Header

image

and visually in the browser we should expect to see

image

Summary

So, in this exercise we accomplished the following

  • Understanding of some of the SharePoint 2013 REST API
  • Usages of that API in Fiddler and the Browser
  • How to Add a SharePoint List Item using the REST API
  • How to create a new LIST using the REST API

This should translate VERY easily into SharePoint Designer 2013 and Visual Studio 2012, that is the point of the exercise. Thats why I am NOT showing how to do it there, I have other post that talk about the Dictionary Object and how to create Headers, just use what I have here and apply it to one of those post and you should be golden. This is me teaching to fish.. 🙂 Cheers.

 

September 3, 2013 Posted by | JSON, Office 365, REST, SharePoint Designer 2013 | , , | 16 Comments

SharePoint Saturday New York 2013 Wrap-up

Praises

Year after year the organizers and volunteers that put on SharePoint Saturday New York City #spsnyc do an amazing job and the intensity and excellence keeps getting better year after year.  Fantastic Job Rebecca “Becky” I, Greg G, Jason G, Tasha S, Tom D, Brandon B, Casey S, and the rest of the crew too many to name.

This year it felt like on all level it came together even with the late lunch 🙂 The audience was exceptionally knowledgeable and engaging, we had fun while learning; i guess it helps that I actually have been seeing them for many years know and we treat each other like extended family.  Even though next year we will be in a new Venue, I will miss the Microsoft facility across form the Hilton where the Halal GUys have their truck.

Regarding my Preso, Deck, Session Information

So, as noted in my presentation, and for those of you that know the way I deliver sessions, I do a lot of demos, hardly any slides. This time however, my session was based on a 2 part blog post I did, I requested that my session attendees not take notes because the blog post will be more information that you had in the session, so please pick it up here

Finally a SharePoint Designer that Developers and BAs will Love – This is the Part 1 of what would have come before the session you sat in and

Part 2 of Finally a SharePoint Designer that Developer and BAs will Love – This is reflective of the session you had

Thank you all for attending, hope to see you again next year.

Cheers.

July 29, 2013 Posted by | JSON, Public Speaking, REST, SharePoint 2013, SharePoint 2013 Workflows, SharePoint Designer 2013, Where is Fabian | , , , | Leave a comment

Part 2 of Finally a SharePoint Designer that Developer and BAs will Love

Purpose of this Blog Post & Primer for this Post

This post follows up on a previous post below where we discussed Workflows in SharePoint 2013 through the eyes of SharePoint Designer 2013 and Visio Professional 2013. Part 1 is linked below, this is Part 2

Part 1 – Stages, Steps, Loops, and Visio Professional 2013

In the above post (this one) I will go about showing you the tooling and various simplistic but apropos examples designed solely to drive the usage home, in Part 2 we will use everything in a practical REAL WORLD sense

Part 2 – Put it all together with HTTP Service Calls via REST and the Dictionary Object/ Variable

In this post I will take a Publicly Available REST API probably the Weather API or Twitter API and use the HTTP Web Request along with the Dictionary Variable in SPD 2013 to surface it in a SharePoint Promoted Links App (List)

What is the Dictionary Object/ Variable

Simply put a Dictionary Variable is an Array i.e. a collection of related items that is indexed/indexable but in more detail it is a container designed to hold a collection of other variables which can be of a different data types. Specifically we have the following actions out of Dictionary

  • Build Dictionary
  • Count Items in a Dictionary
  • Get an Item from a Dictionary

image

Now before we get too deep into The Dictionary Variable/Object, lets take each of these actions one at a time in a simple scheme and see what they do exactly; kick the tires so to speak, in our real world example we will NOT be creating the Dictionary Object as much as consuming JSON data and dumping the JSON data inside a dictionary object. You get it right? JSON data usually comes across as a JSON Array or JSON object, in our case we want that JSON array, even if it is one item, the Dictionary Object/Variable EXPECTS to see an array, I actually wrote a blog post https://fabiangwilliams.wordpress.com/2012/12/31/limitations-when-using-sharepoint-2013-workflows-to-return-json-data/ explaining that issue, which i think is a BUG that should be handled by the SharePoint Team.  But lets leave that on one side for the moment and get back into our intended post.

It is ALL explained Here

As mentioned earlier, a Dictionary Object may be considered a collection of related items. Now to drive this point home lets use what we have at our disposal, in fact we will use two (2) examples, the first of which is to simply build on what we did in Part 1 using the Bank of Fabian Example.  You see, there are ALOT of post out there that do a good job of explaining to you how to get Weather Data, Twitter Data, ITunes Data, you know all the public REST JSON available stuff, and granted I will do one here as well in example number two in the spirit of being complete. However, what you don’t often see is showing how to use SharePoint own REST OData examples, so I will do one here. Lets set it up for you.

Our Use Case

Now I’m a big believer in Use/Business cases when doing Demos’ and not doing parlor tricks for the sake of “Hey! Look at this cool new stuff I can do”, so The idea here is that

1. You can consume data from ANY SharePoint List as long as you have an account that has permission or an App that does, and then act on or manipulate data from inside your Site Collection [assuming a Site Workflow here]

2. You want to expose data/information to an audience that doesn’t have access to certain data or hosted elsewhere

Web Site:  http://adotob.sharepoint.com (this is my Office 365 Dev Site)
List Instance: https://adotob.sharepoint.com/_layouts/15/start.aspx#/Bank%20of%20Fabian

image

So from the above we can tell that we have a list called …/bank%20of%20fabian and it has currently 5 items inside it yes? Ok, good. Now in order for us to get to that data we can call on our SharePoint oData service located here

https://adotob.sharepoint.com/_api/web/lists/getbytitle(‘Bank%20of%20Fabian’)/items

Now you should understand that SharePoint 2013 REST APS doesn’t understand the $json options so, it will render its result in XML, however there are quite a few tools out there that can convert your XML to JSON. In fact if you adjust your request header on your call yourself in Fiddler or in browser extension such as Chrome or FireFox, you can get the visual effect you are looking for as well.  So lets see what happens when you click on the link here for the REST API call

image

Now that is NOT going to be useful to us, so I will use this Chrome extension called ChangeHTTPRequest and after putting in my header i need which is

Accept: application/json;odata=verbose

I get the below

image

Now admittedly that is also difficult to read, so you should invest in the FREE JSON Viewer I always use and its Waaayyy cleaner, just cut and past the entire content and you can see what I mean.

image

So now that we have established some “Trust but Verify” credibility, next lets look at what we need to do in SharePoint Designer 2013 (SPD2013) to move this forward. We will be Demonstrating the following Capabilities in this Example

  1. How to use Dictionary Variable/Object
    1. We will build a Dictionary Variable to hold our HTTP Header information so we can get JSON Data
    2. We will Count Items in a Dictionary Variable so we can get the “Count” value of all items in there to assist us in our “Loop” that we may iterate through all the items in the Dictionary
    3. Get Items from the Dictionary that we may use to (as in our case) Write to the Log History so we can see what we are consuming, but in our Part 2 Example coming up using PUBLIC REST JSON data we will use that data to load into a ListItem
  2. Call a HTTP Web Service
    1. We will call that webservice we discussed above to get oData back
    2. We will discuss the other Dictionary Variables we will need in Support of this Call such as our ResponseContent and our subset of Data we want to inspect
  3. Do a Loop
    1. We will use the total items count that we got from our “Count Items in a Dictionary” Action as our loop counter max value in our Loop construct

Lets Build out our Workflow

The first thing we will do is create a Site Workflow

SNAGHTML1c311bfc

So in keeping with “Fabian’s Best Better Practice” we will stub out our Stages and Get some basic Logic out of the way

image

Once that is done the first thing we will do is build a Dictionary Variable to hold two items we need in order to get JSON data to be returned; we need to set the Accept and Content-Type HTTP header so the browser knows what we expect. We click Action then under Core Actions you will find Build Dictionary, because I use it a lot it is in my Recent Actions also

image

Once you click on ‘Build Dictionary’ you will get…

image

1 – You will then click on ‘this’ which will open up the dialog in [2] and in turn you will click the Add button. In doing so you will enter two Variables into this Dictionary Variable of String Data Type as aforementioned, in [3] I am showing you how i do it for the Accept header attribute.

image

When you are done, click the Ok Button. AND NOW YOU HAVE SEEN HOW TO CREATE A DICTIONARY VARIABLE and an acceptable use of it.

Next we will “Call the HTTP Service” by again clicking Actions in the Ribbon and then…

image

So again by clicking [1] ‘this’ we get the dialog box to enter the information about our HTTP REST Service which in this case we click on the ellipse [2] and put our workflow variable we created earlier that holds our URI, then we click [3] the OK Button.

Oh… i forgot something, but we can do that now… so you see in the above figure the second line has the (Output to Variable:dictionary) right? so I like to tidy up my Variable name, so what I do is click on the word ‘dictionary’[1] and create a NEW variable called RequestHeaders [2]

image

Now that that is done, we want to further configure the Service Request Call so we will ‘Right Click’ on our ‘Call’ and then Click on Properties..

image

Then I want to set the “RequestHeaders” Property to my Dictionary Variable “RequestHeaders”

image

and then click the OK Button.

Next we will need to ALSO create another Dictionary Variable to actually HOLD the data that’s coming back, if I haven’t said it before, that data comes back in the form of a JSON Array and the Dictionary Variable is the only object suitable to accept that data. So

image

and we name it ‘JSONResults’ so if we inspect the properties again you will see that we have everything we need now to call out to our HTTP Service and get meaningful results back

image

So now we can actually got GET the DATA now, by using yet another Action in the SPD2013 Ribbon, this time its the ‘Get Item from Dictionary’ Action

image

Once we do that we will want to click the ‘item by name or path’ link and give it the SUBSET or WHOLE path to the data we inspected in our JSON Viewer or whatever tooling you have to determine what you need. In our case we want the “d/results” subset then we will use the Index variable to get each line item. Furthermore we will want to store this information AS WELL into another Dictionary Variable.

SNAGHTML1c5cf00d

So in [1] you have me setting the path based on what i want from evidence in the JSON Viewer, then in [2] I am showing you the relationship of use getting the values from the HTTP Call made earlier and finally [3] I’m dumping that data into another Dictionary Object.

SNAGHTML1c5eb07e

if you forgot here is what i need to build that path take a look above at [1], then [2] to get my Array. In [3] I will use that below using the Index Variable inside a Loop Construct.

So that brings us to the last unique item we will be doing in our Dictionary Item and that is to get a COUNT of all the items in the Dictionary We Need. So again we go to the SPD2013 ribbon

SNAGHTML1c6355f1

and we now need to tell the action which Dictionary Item to count, REMEMBER we want the one that we did the Subset of, not the one returned from the Service Call.

SNAGHTML1c65a1a0

Above [1] we are referencing our Subset Data and we are creating a new Variable of Integer type [2] to put that Number in. NOW this is an important distinction as well because what we have here is our UPPERBOUND number in the Array for out Loop Counter 🙂 smart huh LOL

Next we will enter the Stage “Loop and Write out our Data we Need” and put a Conditional Loop inside there that will use the Index value of 0 to start and STOP when we have reached the UPPERBOUND value of items in the array

image

Now when that’s all done, WE are done, one more Screen Shot and some guidance and you can take a mental break..

image

Now admittedly its a bit busy, and yes, its been hours now since I started on this blog and fatigue is setting in 🙂 but I think i can capture everything I need to say here, and I will wrap up as usual with a video. So we are setting the Initialize and Upper bound of our Loop counter in [1] and [2] respectively. In [3] we are actually going to WRITE OUT TO THE LOG the information garnered from the above two lines and the remainder is the Loop Counter which you saw in Part 1, so now time for the video of it all.

So this blog post is long enough, what I will do for the External Data Call is do it in another Post and I will Link to it here. Hope you learned something new.

Cheers

Up Next – Addendum to Part 2 showing an External HTTP Service Call using REST/JSON [Hyperlink forthcoming]

April 29, 2013 Posted by | JSON, SharePoint 2013, SharePoint Designer 2013, SharePoint How-To, Webcast | , | 3 Comments