Trick or treat? Group items by month

OrderedMonthsIt’s that time of the year again when the unnatural becomes the norm, so let’s continue the tradition started last year with the SPELL program. Our goal today will be to display list or library items grouped by month, as shown in the screenshot.

The deal is that we don’t want any custom code or workflow here, just the regular out of the box UI features. We’ll be creating two calculated columns, called Year and Month, where we’ll insert the appropriate formulas. I chose the “Modified” column for my sample formulas, but of course any other date column would work too.

The YEAR and MONTH functions

YearNumber

A quick review of the SharePoint date functions gives you YEAR and MONTH that should fit the bill:

Year:  =YEAR(Modified)
Month: =MONTH(Modified)

The result is less than satisfactory though (see screenshot), as the year is displayed with a thousands separator. and months are displayed as numbers.

The magical TEXT function

MixedMonthsFurther exploration will take you to the TEXT function. It is not very well documented in SharePoint, fortunately you can rely on the Excel documentation and come out with the following formulas:
Year:  =TEXT(Modified,"yyyy")
Month: =TEXT(Modified,"mmmm")

Still not happy with the result? Right, the months are displayed in alphabetical order, not sequential order, not yet an ideal experience for our end user.

So let’s pull our last trick, and use the following formula for the calculated month:
Month: =REPT(" ",13-MONTH(Modified))&TEXT(Modified,"mmmm")

You can see the final result live on this page.

What’s the trick? We are still relying on the out of the box alphabetical sorting, but to force the order, we are adding a bunch of white spaces before the month name. The calculated Month actually contains the following values (each _ represents a white space):
____________January
___________February
__________March

__November
_December

Now, why don’t we see these spaces on the Web page? What makes the magic work is that when you insert multiple spaces in a Web page, the html specification says that

user agents should collapse input white space sequences

That’s it!

If you want to get really fancy, you could even use the zero-width space character. The best part is that people who edit your formula won’t even understand the trick, as the zero-width space won’t be visible (there’s however a good chance that they break your cool formula).

To take this further

You can apply this trick to other situations. A typical example is a color code. The alphabetical order will give you Green-Red-Yellow or Amber-Green-Red, you can address that by adding the appropriate leading spaces.

Be careful with this technique though: even if the rendering looks fine, the spaces are indeed stored in the field, and this might break other customizations. So this trick is better kept in a calculated column that will be exclusively used for rendering purposes.

Advertisements

HTML Calculated Column + Client-Side Rendering

Five years after the first release, the HTML Calculated Column remains the most popular topic on this blog.

The original page has been visited more than 200,000 times. It is definitely outdated, and in recent years I have pushed several new variations of this technique. The most popular is the color coding solution posted in the SharePoint User Toolkit, backed by this tutorial.

The most frequent issue reported by users has been the upgrade from SharePoint 2007 to SharePoint 2010. This is actually all taken care of in the above links… but you need to read the fine print.

I plan to rewrite the instructions, especially as in the meantime Microsoft has pushed another version of SharePoint, and the SP 2010 update… doesn’t work with SP 2013! Well, there’s actually a simple fix for SP 2013, and “Panoone” posted it as a comment a couple days ago (@Panoone: thanks again! And let’s get in touch to discuss this further).

But that’s not all. SP 2013 brings a bunch of new client side technologies, and one of them works very well in our case: Client-Side Rendering.

What is Client-Side Rendering?

JSLinkEditYou can use Client-Side Rendering (CSR) in SharePoint to manipulate the rendering process of list views. Does this sound familiar? That’s exactly what my HTML Calculated Column has been doing for years! Except that now it is an official component integrated with list views. When you edit a Web Part, the very last option is the JS Link placeholder (see screenshot).

It will certainly take several weeks before I find time to update the SharePoint User Toolkit, so for those of you who are already familiar with both CSR and the HTML Calculated Column, let me share the code for SP 2013.

In the code, Calculated is the name of the calculated column.

var ccContext = {
  Templates: {
    Fields: {
      "Calculated": {"View": "<#=STSHtmlDecode(ctx.CurrentItem.Calculated)#>"}
    }
  }
};
SPClientTemplates.TemplateManager.RegisterTemplateOverrides(ccContext);

So easy, just 200 characters! STSHtmlDecode() is a JavaScript function provided by SharePoint that allows the conversion of the html string.

Of course, you could avoid using a calculated column, and build the html directly in the JS Link file. But then each rendering would require a different file. The beauty of the HTML Calculated Column is that one single JS Link file can support all the views in your site.

A warning!

Some time ago, I wrote about security risks when you use the HTML Calculated Column. As far as I can tell, the same warning applies to Client-Side Rendering. Handle with care!

CSR references

When Microsoft released SharePoint 2013, the documentation on Client-Side Rendering was quite poor. Fortunately SharePoint bloggers stepped in and you can find some nice posts to get familiar with CSR. Wes Preston’s blog is an excellent start.

So is the old way dead in SharePoint 2013?

Not quite yet! Not every view accepts Client-Side Rendering, and for example you’ll still find my tutorial helpful for calendar views.

More pocketSOAP examples: Pie and Bubble Charts

A couple weeks ago, I introduced pocketSOAP, an experimental JavaScript library that facilitates interactions with SharePoint SOAP services.

Today let me share two more examples built on SOAP and the templating engine, combined with Google Visualization Charts, to render charts from SharePoint lists.

If you have subscribed to the SPELL newsletter, you’ll find pocketSOAP in your mailbox this week. Keep in mind that it is an experimental library, only the full SPELL library is meant for production environments (cf. disclaimer in the previous post).

I am not expecting you to be impressed by the screenshots below. After all, I’ve been publishing client side charting demos in this blog since 2008, and since then many others have followed. The real kicker here is how compact and versatile the code is. Even if you are not interested in client side scripting for a production environment, a tool like SPELL/pocketSOAP could prove really useful in a prototyping phase, and save you many hours of hardcore (re)programming!

Example 1: Pie Chart
PieChart
For this first example, I used the same SharePoint list as in the previous post, with two columns:

  • the default “Title” for the item name
  • “Priority” as a choice field with 3 options (high/medium/low)

The code:

<div id="results"></div>
<script type="text/javascript" src="pocketSOAP.min.js"></script>
<script type="text/javascript" src="//www.google.com/jsapi"></script>

<script type="text/javascript">
// Load the Google charting library
google.load("visualization", "1", {packages:["corechart"]});
// Get items from the Projects list
var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestZone",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"Projects",
 viewFields:"<ViewFields><FieldRef Name='Title' /><FieldRef Name='Priority' /></ViewFields>"
});
pS.when(promisedItems)
.then(function(request){
 // When ready, run the charting function
 google.setOnLoadCallback(function(){drawChart(request);});
});
 function drawChart(request) {
 // The templating engine formats the data string according to Google format
 var dataString=pS.applyTemplate({
 template:'[["Project","Priority"][<|,["|Title|","|Priority|"]|>]]',
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z")
 });
 var dataTable = google.visualization.arrayToDataTable(JSON.parse(dataString));

// Group by priority. Priority is the second column (index = 1)
 var data = google.visualization.data.group(dataTable, [1], [{'column': 1, 'type': 'number', 'aggregation': google.visualization.data.count}]);
 var chartOptions = {'title':'Priority','width':400,'height':300,colors:['red','#FFC200','green']}; // #FFC200 is the color code for amber
 var chart = new google.visualization.PieChart(document.getElementById('results'));
 chart.draw(data, chartOptions);
}
</script>

The SOAP call is almost the same as in the previous post. The only difference is that I have explicitly told SharePoint which fields I want to retrieve, via the viewFields option. This is not mandatory but it will reduce the size of the dataset and improve performance.

The use of the templating pattern (pS.applyTemplate) to convert data formats is unusual – as a matter of fact I have never seen this used elsewhere. I am not advertising it as a best practice, and more traditional object manipulation could be used here to the same effect. But I really like how templating makes the code easier to maintain.

Templating is a very popular pattern in modern JavaScript, and can be found in many libraries, like jQuery, mustache, handlebars and others. The SPELL/pocketSOAP flavor might not be as powerful as those big names, but in just 1 kb of code it still has a lot to offer, and – as you would expect – it is well adapted to work with SharePoint Web Services, for both XML and JSON output (to use it with JSON instead of XML, simply replace the [< >] notation with [{ }]).

Example 2: Bubble Chart

BubbleChart

For this second example, I created a SharePoint list with 5 columns:

  • the default “Title” for the product name
  • “Risk” as a choice field with 3 options (low/medium/high)
  • “Cost” and “Revenue” are numbers
  • “Profit” is a calculated column:
    [Profit]=[Revenue]-{Cost]

The code:

<div id="results"></div>
<script type="text/javascript" src="pocketSOAP.min.js"></script>
<script type="text/javascript" src="//www.google.com/jsapi"></script>

<script type="text/javascript">
// Load the Google charting library
google.load("visualization", "1", {packages:["corechart"]});

// Get items from the PMO list
var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestSite",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"PMO"
});

pS.when(promisedItems)
.then(function(request){
 // When ready, run the charting function
 google.setOnLoadCallback(function(){drawChart(request);});
});

function drawChart(request) {
 // The templating engine formats the data string according to Google format
 var dataString=pS.applyTemplate({
 template:'[["Product","Cost","Revenue","Risk","Profit"][<|,["|Title|",|Cost|,|Revenue|,"|Risk|",|Profit|]|>]]',
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z"),
 sanitize:function(string){return string.replace(/float;#/,"");}
 });

var dataTable = google.visualization.arrayToDataTable(JSON.parse(dataString));

var chartOptions = {
 title: 'PMO',
 width:500,
 height:400,
 colors:['green','#FFC200','red'], // #FFC200 is the color code for amber
 legend:{position:'top'},
 hAxis:{title: 'Cost',maxValue:100},
 vAxis:{title: 'Revenue',maxValue:100}
 };

var chart = new google.visualization.BubbleChart(document.getElementById('results'));
 chart.draw(dataTable, chartOptions);
}
</script>

Note in the template the difference between text fields (for example “|Title|”) and number fields (for example |Cost|). Also, this time I had to include a sanitize function to remove the weird “float;#” string that SharePoint prepends to calculated columns.

The above code samples could easily be tweaked to work with other client side charting libraries, like Dojo, HighCharts, or [name-your-own]. If you are interested in a demo, leave a comment with the name of your favorite charting tool!

Teaser: pocketSOAP, an ultra lightweight library to interact with SharePoint SOAP services

[6/13/2013] Update: I have changed the terminology, from Future back to Promise, to conform to the living standard.

Disclaimer: the solution presented in this article is part of SPELL, a coaching program offered by User Managed Solutions LLC. Access to the full set of SPELL solutions requires a paid registration. However, the program also includes free evaluation samples like pocketSOAP, all you need to do to get them is show your interest by registering to the SPELL newsletter.

The SPELL newsletter currently has ~180 subscriber companies. The issue featuring pocketSOAP is scheduled for the first half of June.

For years, the SharePoint SOAP services have been a favorite among power users and front end developers. Their popularity is due to an accumulation of factors:

  • They are supported across all SharePoint versions
  • They offer a wide range of features, from simply accessing list items to building sites or managing permissions
  • They work in client side environment, and don’t require server side access.
  • They are well documented, thanks in particular to the work of SharePoint bloggers (cf. references at the end of this post)

Since the SP 2010 release, Microsoft has been embracing more and more the client side, and has pushed additional tools like CSOM, REST (listdata.svc, _api), and client side rendering for list views (JSLink). At the same time SOAP services have been deprecated (but are still supported in SP 2013). Developers are faced with a difficult choice: they’d like to adopt new technologies, but at the same time these technologies don’t offer (yet) the same coverage as the legacy SOAP services. And of course they don’t work in older versions.

The purpose of pocketSOAP is to showcase how I dealt with this dilemma in SPELL. When you want to interact with half a dozen interfaces, as SPELL does, you can’t afford a 100 kb footprint just for one! So here we go, 5 kb of fully functional code, no dependency! The list of pocketSOAP ingredients:

  • SOAP: ~ 1 kb
  • Ajax: ~ 0.4 kb
  • Promises: ~ 1 kb
  • Templating: ~ 1 kb
  • Various helper functions: ~ 1.6 kb

A simple example

Note: pocketSOAP targets developers and assumes that you are comfortable with JavaScript patterns such as ajax, promises and templating. If you are an end user, you might be more interested in user friendly samples like the SPELL Tabs or the SPELL Charts (and of course the SharePoint User Toolkit).

ProjectColorFor my simple example,  I have just created a custom list called “Projects”, with 2 columns:

  •  Title is a text field for the item name (Project 1, Project 2,…)
  •  Status is a choice field for the item status: green, yellow or red.

Expected result (cf. screenshot): color coded indicators that displays the health of each project.

The code (pS is the shorthand for pocketSOAP):

var promisedItems = pS.soap({
 // Service info
 site:"https://usermanaged.sharepoint.com/TestSite",
 service:"Lists",
 operation:"GetListItems",
 // Service parameters
 listName:"Projects"
});

pS.when(promisedItems).then(function(request){
 // Apply template
 document.getElementById("results").innerHTML=pS.applyHTMLTemplate({
 template:"<table>[<|<tr><td><div style='width:12px;height:12px;background-color:|Status|;'>&nbsp;</div></td><td>|Title|</td></tr>|>]</table>",
 prefix:"ows_",
 data:pS.byTagNS(request.responseXML,"row","z")
 });
});

How the code works:

  • pS.soap() handles the SOAP request (ajax). It needs two sets of options: information about the service itself (in our case Lists/GetListItems) and parameters specific to the call (in the case of GetListItems: the list name, and maybe specific filtering options and a shortlist of the fields to retrieve).
  • pS.when().then() is a promises pattern. It ensures that the SOAP request is completed before we move to the next step.
  • pS.applyHTMLTemplate() runs the templating engine, in this specific example to output a html string. In the template expression, [< >] means that we are dealing with a collection of nodes. If we were using the new REST interface and retrieving JSON, we would simply use  [{ }] instead of [< >].
  • the prefix ows_ is just here to address an oddity with SharePoint Web services, where “ows_” is prepended to each field name. So for example to get the Title field, you need to retrieve the attribute called ows_Title.
  • the data is the result of ajax xmlHttpRequest. If you are familiar with the SOAP service GetListItems, you know that each item is returned as a namespaced “z:row” node.

How many SOAP services does pocketSOAP support?

I don’t know. So far I have only tested it for half a dozen services representative of my scenarios (90% of real life use cases are about getting or updating list items). I plan to document this as I and the SPELL followers continue testing more services.

Can I use SPELL/pocketSOAP to aggregate values from multiple lists?

Yes. Thanks to the promises pattern, the script can collect data from multiple lists in parallel, then trigger the rendering when all the items have been collected. The pocketSOAP documentation will include such an example.

Note however that for a large number of lists, client side code might not be the best approach and you’ll want to consider other techniques.

How do I make a synchronous call?

You can’t, this option has been removed from SPELL last year. The choice is to embrace a modern JavaScript approach where remote calls are asynchronous (the first “a” in ajax), and dependencies are managed via promises: when(this is completed).then(do that).

In SPELL, not only ajax calls, but also script loading, iframe loading, document ready, and some other operations are treated as promises.

Did I go too far with pocketSOAP?

The answer is… yes! If you play with pocketSOAP, keep in mind that it is a laboratory rat. The SPELL library – the production version – offers much better balance between the load effort  and efficiency at runtime.

How is SPELL different from pocketSOAP?

1/ SPELL is for production, while pocketSOAP is an experiment. For example error handling has been reduced to a minimum in pocketSOAP.

2/ SPELL supports half a dozen interfaces, SOAP just being one of them. Apart from SOAP ($P.soap), it also relies on the RPC method ($P.rpc), the REST services ($P.listdata and $P._api), etc.

3/ SPELL offers not only core features aimed at developers, but also end user solutions like the SPELL Tabs and mini-BI.

How come SOAP has never been mentioned before on Path to SharePoint?

Until recently, I have always considered that SOAP was a heavy solution and should only be used in large scale implementations, or when everything else fails. For  the samples showcased in this blog and in the SharePoint User Toolkit, my preference went to the RPC method, a more straightforward and end user friendly approach.

However when I started building SPELL two years ago, SOAP was an essential building block in a projet of that size. The decision to include SOAP was even easier after I reduced the code size to just a couple kb!

References

The official Microsoft reference for SharePoint 2013 Web services can be found here.

The first to publish a JavaScript library to wrap SOAP services was Darren Johnstone in 2008. His original site doesn’t seem to exist anymore, to get the files try an internet search for “SPAPI”.

SharePoint superstar Jan Tielens published in 2009 a series of posts to showcase the use of SOAP services in conjunction with jQuery.

That same year, Marc Anderson released SPServices. This remains to date the most useful and comprehensive resource on the subject. When people look for documentation on the SOAP services, I usually point them to Marc’s Codeplex site rather than the official Microsoft documentation.

SOAP services are also a favorite of Alexander Bautz, and he uses them in some of his published work.

Alexander’s blog drove me to another library called SharePointPlus, developed by DELL employee Aymeric Kodono and released under a GPL v2 license.

There are certainly others I am not aware of. For example I haven’t tested this framework that seems to deal with the SOAP services (and btw is not maintained anymore). If you have more information feel free to post it in the comments!

Pie and Bar Charts (Google connector)

Pie

I already mentioned it briefly in an earlier post: the SharePoint User Toolkit now includes a tool that allows you to add simple pie or bar charts to your SharePoint pages.

The code is actually a connector that points to the powerful Google Visualization interface. For this reason, Internet access is required.

A common concern with online tools is data security. In this case, we are safe, Google’s privacy policy explicitly states that no data is sent to the server. The charts are directly rendered in your browser. This is a major difference with the Google Image Chart API, where data is sent in clear to the server, which uses it to build and return an image.

Under the hood, the code works the same way as the Image Rotator: it retrieves data from a specific view of your SharePoint list.

Live demo: countdown/countup

Countdowns, and generally speaking comparisons with Today’s date,  have been a recurring theme on my blog.

SharePoint 2010 hasn’t brought much improvement to the “Today” issue, so the workarounds published on my blog in 2008 remain a good reference, either the one I wrote for Data View Web Parts or Alexander Bautz’ follow up article for list views.

A year ago, I blogged about two other solutions, one relying on jQuery, the other on Flash and ClockLink.com (the demos are not active anymore).

Today I am showing two new examples that are much simpler than the previous ones. The live demo is here:
http://sp2010.pathtosharepoint.com/Portfolio/Pages/Countdown.aspx

Why much simpler? Because the only thing you need in the page is my Text to HTML script – the exact same that is used for color coding calendars. The calculated column will take care of all the rest (for example the formula to calculate the difference between target date and Today, or the color selection).

Note that there’s a key difference between the two examples: the “Countdown” column relies on the local time of your computer, while the “TimeAndDate” column pulls today’s time from TimeAndDate.com. In the past, I have already highlighted this important choice (for example, don’t use the local computer time if you’re building an auctions site!).

The drawback of this new approach is redundancy – the current date is calculated for each item. So I would recommend to only use it for views with few items.

You’ll notice that the column filters also render the HTML (cf. above screenshot). That’s because the page uses a custom version of my Text to HTML. It is still work in progress, so please don’t copy this script!

A new location for the SharePoint User’s Toolkit

The SharePoint User’s Toolkit has a new official page:
http://sp2010.pathtosharepoint.com/SharePoint-User-Toolkit/

The old location will remain active but won’t be updated anymore.

The new site is based on SharePoint 2010 and hosted by fpweb.net. Thanks to the support of fpweb.net, I expect the new site to be more reliable and offer a better user experience.

The SharePoint User’s Toolkit is a collection of tools designed to help end users build advanced customizations. It includes for example the Easy Tabs and an Image Rotator. It will continue to grow, with new tools added every month.

Regular users of the Toolkit will notice that several solutions are not in beta anymore. I haven’t actually made any changes to the code, the beta versions are becoming official simply because no issue was reported in the past few months.