Items Storage, the cookies’ big brothers! (+ cross domain trick)
pujoljulia July 7, 2016

The cookies are not the only way to save users’ data at the browsers, there are many other methods to do it. After cookies, the most relevant tool to store and hold data it’s called Item Storage or just Storage, which has great advantages as well as big downsides compared to the cookies.

A good new is their storage limit, while cookies are able to store up to 4096 bytes per domain, item storage is oscillating between 2 and 10 mega bytes (depending on the browser used) however it’s advised to not exceed 5MB (5.000.000 bytes) per domain. Note the existent huge difference between 4096 bytes for the cookies compared to 5.000.000 bytes for Item Storage.

For make this figures more realistic, take 2 bytes as 1 character (strings in javaScript are UTF-16, so each character requires two bytes of memory), which results up to 2.5 millions of characters per domain (folks… we can store here the entire user’s live if we want!). Item Storage also has good browser compatibility.

Another good point is their client-side based nature, unlike cookies the items don’t travel from the browser to the server each time a petition (HTTP request) is made, which improves the page’s load times. More good stuff related with Items Storage comes at time to get, set or remove them, as the browsers incorporate an exclusively API to manage them. Here are the main API’s methods:

  1. setItem(name, value): Stores on the user’s browser an item with ‘name’ name and ‘value’ value.
  2. getItem(name): Get the item’s value given an item name.
  3. removeItem(name): Remove from the user’s browser an item with the ‘name’ name.
  4. clear(name): Clear all items stored on user’s browser.
  5. key(position): Get the item’s value given an item position.

So far everything seems good but as I said, there are several disadvantages respect the cookies as for example, there is no way to attach them an exact expiration date (items expire either when browser is closed, called Session Storage, or never, called Local Storage) and as I also mentioned, they cannot be managed by server-side languages, which makes the things difficult for advertising guys.

But the real downside comes with the cross subdomain management. When I write about the cookies, I already showed you how establish a cookie to domain level in order to be managed (read, modify or delete) across subdomains of de same domain (for example between www.pujoljulia.com and analytics.pujoljulia.com).

Sadly Item Storage doesn’t allow to read ‘items’ between subdomains and that is the reason for what we are here. What I’m going to show you on the below sections is how make Item Storage to work across subdomains and domains as well as automatize the whole process into the Data Layer. Next sections are gonna be very interesting, that I can assure you!

Show me this big cookies called Item Storage!

Sure, sure, there is no best way to pick up a concept other than see it. Open the console (Alt+Cmd+i on Mac devices and f12 on Window devices) and click at Resources’  ‘Application’ tab:

item-storage

At the left you can see all tools the browsers dispose to save information. Observe that in case of Item Storage, there are two types: Session Storage and Local Storage. The unique difference between them is that all items from Session Storage expires when the navigator is closed (that does not mean the tab closed, means exactly the navigator completely closed) and those from Local Storage never expires (unless we delete them!).

All of above explained methods (setItem(), getItem(), removeItem()…) can be used for both Session and Local Storage. The way to distingue them is by just invoking the appropriate object of the window, so ‘sessionStorage’ object for Session Storage and ‘localStorage’ for Local Storage. Let’s set up an item for each one, type the next code at the browser’s console:

And take a look again at ‘Resources’  ‘Application’ tab of the browser’s console:

Easy right? Now you can start to playing around with the other methods to delete (seessionStorage.deleteItem() or localStorage.deleteItem()) or get the values (sessionStorage.getItem() or localStorage.getItem()), it’s as easy as with the setItem() method used above.

But this is not enough, isn’t? Probably your page or your client’s page has more than one subdomain or even more than one domain and also probably there are embed iframes from other domains or even frames that point to other subdomains. Usually the websites are a mess, for that reason we have to devise some mechanism to manage the items regardless the domain or subdomain where the pages are. Let’s jump to the next section!

Why bother using Item Storage?

Basically because its storage limits, but let me brief you about a real use case. Time ago I used to work for a car brand which obviously offers a car configurator in their website to help the users to get an idea about the car desired. While I was designing the Adobe Analytics’ implementation it occurred to me to save into strings all user’s interactions (strings like features added, features removed and cars selected) on the configurator in order to figure out how it was used (by saving all information to Adobe Analytics) as well as improve the user experience (in real time) based on user’s interactions (through behaviouring and A/B testing tools).

Save information for analytics purposes was easy, as I took advantage of the Adobe Analytics’ props variables. For each hit I sent the proper information to the tool and it took care to construct the string inside Adobe Analytics but… what happen with improving the user experience in real time? What if I want to show a pop up when a user has followed an specific path? What if I want to open a help-chat when, by the user’s interactions, seems the user is a bit lost?

The first idea that occurred to me was to use cookies, but you rapidly can figure out how easily a user which is playing around with the car configurator can reach the storage limit of cookies, so yes, I broke the website during my tests. Then googling I found out about Item Storage, so I decided leverage on it for my purposes but then again other problem arose: the car configurator was in a different subdomain. Oh well, I was fucked up again! (sorry for the bad language) but… there is always a workaround right? So eventually, after googling again, I found this nice library and I decided to adapt it into the Data Layer to play with the items as much as I wanted through the Tag Management, it’s not great?

Cross subdomain/domain Item Storage

Tech Time! This part is gonna be tough my friend, but don’t hesitate, all the necessary information you need to get the point is in this post as well as in my post about iframes (which you shall read before continue). Ready? All right! First what I’m going to do is to draw my plan to share Items across subdomains or domains, take a look:

itemStorage-cross-domain-subdomain

Not bad ah? Basically I’m going to follow the above 5 steps to get my goal and therefore share items across subdomains/domains. Before to go through all the steps is necessary to take an important decision: which domain/subdomain is gonna store all items?

As you can imagine by the image above, the trick for share items among subdomains/domains is by embedding an iframe, so before to start you have to decide ‘who is going to be iframed’. If your site is composed by different domains and subdomains, my advise is to choose the one where the users remains more time or/and you expect more activity on, as probably will be the one where to track more stuff. Take a good decision will reduce the calls between subdomains/domains.

The example which I’m going to go through in order to explain all the steps is pretty simple: the domain analyticstestlab.com takes the ‘main domain’s role’ and pujoljulia.com takes the ‘secondary domain’s role’ which basically means pujoljulia.com is going to contain an invisible iframe that points to analyticstestlab.com. Confused? Let’s start from scratch:

1. The basic material: Data Layer and Tag Management on site!

It was obvious that the first step would be to place the Data Layer and Tag Management scripts on both sites (pujoljulia.com and analyticstestlab.com), which I guess is something you already have done in your sites. Next thing is to automatize a process to manage the cross domain items and that is the kind off stuff the Data Layer should do, so let’s set up some helper methods to accomplish that.

2. The manager: crossdomainItem() method

As like I did with the cookies, there will be basically three methods to manage items: one to set up them (setItem), other to read them (readItem) and finally other to delete them (deleteItem). Remember that we’ve to build this methods because the default ones explained at the beginning of the post can only share the items among the pages under the same subdomain.

Cool stuff! but wait wait… somehow we have to implement an iframe and manage the messages between iframes and pages right? And… we are not going to repeat those actions for each one of the mentioned methods, ain’t we? Of course not, we are PRO, and as a such we’re going to create a new method (crossDomainItem) that it’s gonna take care of these stuff, which it’s going to be used for the others three methods (set, get and delete ones), something like that:

data-layer-cross-domain-method

Very good! Let’s see the javaScript code that the crossDomain(); method contains:

Do not freak out! Let’s go step by step:

  • (1). The first block of code it’s just defining some local variables that I’m going to use on the method. The ones which you have to focus by now are the first ones (g and h). This two variables are used to define where the iframe has to point to, so in my case to http://analyticstestlab.com/iframe.html (g + h), which must to be changed by your iframe’s page. Click on the URL’s link and take a look on that page… nothing right? Well, that is not true, there is something, inspect the source code of the page. Surprise! This is an important stuff, but the explanation will come later.
  • (2). Second code’s block is quite simple, it’s just checking if the browser is compatible with all javaScript commands that the method is using. If some incompatibility is found the method will stop without provoke any error, however rest assure that more than the 90% of your users will use browsers that will be compatible with my code.
  • (6). Ou Ou! why you jump straight away from 2nd block to 6th? Well because it will make more sense for you! So this block is really important as it does three important things:
    1. Set up the necessary invisible iframe to establish the connection between domains.
    2. Set up an event listener to the iframe to call a ‘callback’ (callback in this case is the ‘i();’ function) when the ‘load’ event is dispatched. The ‘load’ event it’s gonna be fired when the page inside the iframe (in my case http://analyticstestlab.com/iframe.html) be completely loaded. The similar code on below it’s just for earlier IE versions.
    3. Set up an event listener to the main page (this page) to call another ‘callback’ when the ‘message’ event is dispatched. The ‘message’ event is gonna be fired when the iframe’s page (analyticstestlab.com) sends a message to this page (pujoljulia.com). The similar code at below it’s just for earlier IE versions.
  • (3). Thanks to the event listener attached to the iframe, now the ‘i();’ callback will be executed once the iframe be completely loaded. What the ‘i();’ callback does? Just a couple of things:
    1. First is setting up the digitalData.storage.iframe as true. Probably now it does not makes sense for you, but soon will do, for now just keep in mind that the Data Layer is aware when the iframe is loaded.
    2. Next the callback goes through digitalData.storage.queue, which contains all items’ actions we’ve ordered to do through the setItem() or get(item() or deleteItem(). Think of it like when you make your online grocery purchase, you add to the basket all desired products and eventually you send the order, then when the supermarket is ready it sends the deliver driver to bring you all the stuff. Taking the parallelism, digitalData.storage.queue is the ecomerce store’s basketMakes sense right? If not, it will do, be patient 😉
  • (4). This block executes the ‘m();‘ function (remember it’s called by the callback ‘i();‘ as many times as orders digitalData.storage.queue contains). Basically this function makes the postMessage() call to the iframe, meaning, is sending the message to the iframe (http://analyticstestlab.com/iframe.html). The order (n.request) is an object composed by different properties (properties that tell the iframe what to do, like ‘please set an item’ or ‘please delete it’) which thanks to JSON.stringify() method are converted to string, as use to be better send the message as string instead of as an object. The content of the request (n.request) will be explained later. Finally ‘g’ is a variable that contains the iframe’s URL, which should be changed by your ‘iframe main domain’.
  • (5). In the (4) block the message has been sent, but what happens when we ask to the iframe for the value of some item? We need to receive it, right? That’s why it’s necessary a function to receive the response from the iframe, and that is precisely what ‘p();‘ function does. Like we’ve used JSON.stringify() to convert the object to string in order to send the message, now to read it we’re going to use JSON.parse() to do the opposite. Once the message (the value of the item) is received, it is sent to another callback. Why the value is sent to a callback instead of store it into Data Layer’s variable or be returned as an outcome of the getItem() method? Well, this requires you to read about asynchronous javaScript, but in a nutshell, what happens is that ‘p();’ function doesn’t really know when the iframe is gonna reply (as it depends on how long will take to the iframe to make the response), so p(); is basically telling us ‘Hey! I don’t know when the iframe is gonna reply, but when he does, I will leave the message into the callback’.

Notice that at the end, the method returns ‘d’ and ‘m’ variables/functions, this is because both are going to be used by the others methods explained at the following tabs.

3. The workers: setItem(), getItem() and deleteItem() methods

These are the methods that you really are going to use through the Tag Management, as each time you want to remember some value took it form the Data Layer or straight away from the page, just using above methods you will be able to set, get or delete them as an items. Now the things will start to make sense for you. As usual, first let’s show the methods’ code:

Notice that all three are very very similar, that will simplify the explanation. The first thing the methods do is just check if the required parameters are present (remember that parameters are placed between the parenthesis when a function is called, for instance dataLayer.setItem(‘parameter1′,’parameter2′,’parameter3’)), if so then the crossDomainItem(); function is called and the results are stored at ‘u’ variable. Remember that crossDomainItem(); returns ‘d’ (which checks JS compatibility) and ‘m’ (which sends the messages/orders to the iframe).

After that, the methods check the JS compatibility, if it is okay, then the message/order is built and eventually stored at ‘w’ variable. The message/order is obviously different for each method, as the goal of each one is different. Here is the list of all message/order components for all three methods:

  • id: Used to maintain the order on the queue.
  • type: Tells to the iframe the ‘action’ to execute (set, read or delete).
  • name: The name of the item where the ‘action’ (type) will be executed.
  • value: Used by setItem() method. Gives a value to the item.
  • expiration: Used by setItem() method. Gives an expiration to the item (if not informed the value expires when browser is closed, ‘never’ value indicates that the item will never expire).
  • callback: Used by readItem() method. Execute the callback when the value is returned by the iframe.

The last line of code of the methods checks if the iframe is loaded, if so sends the message/order straight away, otherwise push it to the queue. This check is really important since you can call this methods several time before the iframe has been loaded so there have to be a way to ‘save on memory’  the messages/orders before everything be ready to send them.

Good so far, let’s test them! Fist of all let’s create an item. Open the console and on console’s tab of the console place the next code:

Then move to ‘Resources’  ‘Application’ tab and look at the resutls:

local-storage-cross-domain-iframe

Indeed the item has been stored but… this is the same we’ve done at the beginning of the post right? Mmm no, is slightly different, look at the highlighted domain, it is not the domain where you are reading my post, right? Of course no, it’s the iframe’s domain, so that means the method has done its job: From the pujoljulia.com domain stores an item to analyticstestlab.com, or with another words, cross domain Item Storage.

Fantastic! But now of course we want to read the item, isn’t it? Okay, let’s apply the readItem() method:

And let’s see the result:

read-cross-domain-item-storage

It works! So the method has asked to the iframe about the item’s value and the iframe has returned the value but… the second parameter (function (value){console.log(value)}) is weird right? Remember the callback? Well so this is the callback, which means the value can only be handled inside the function, so whatever thing you want to do with the returned value, ‘in theory’, have to be done inside the function.

And finally the remaining function is the delete one, so let’s delete the item:

And if we check back the ‘Resources’  ‘Application’ tab of the browser’s console:

delete-item-storage-cross-domain

Voila! The item has disappeared! Easy right? Just placing the methods to the Data Layer (click here to see the complete Data Layer along with all methods discussed on other posts) you can manage cross domain items as many times as necessary!

But… we’re missing a point, have you wondered what the iframe does to reply to the pages and therefore store, get or delete the items? Click on the next tab my friend!

4. The storehouse: The iframe's domain

Remember that we’ve already check the analyticstestlab.com page that contains the script which where the iframe points to : http://analyticstestlab.com/iframe.html. This is the code the page contains:

Wow it’s a long piece of code! In this case I’m just going to highlight the important parts:

  • The most important thing is the ‘whitelist’ variable. This is a comma separated array which stores all domains that are going to be able to access to this page, so right now only pujoljulia.com can activate the script’s page to reply the postMessages(), but if for example I add here ‘nike.com’, the nike.com domain could also set up items at analyticstestlab.com. Be vigilant and add just these domains you trust and you need them to access to the iframe’s domain..
  • mbSizeLimit is also an important variable since it determines the maximum size of MB/characters the methods are going to allow. I recommend you to leave the value at 0.01 (2500 characters/5MB) since as I said at the beginning of the post, is the recommended size. If for some reason the size is reached, all items are going to be deleted for security reasons.
  • Notice that at the end of the code, a ‘message’ event listener is attached in order to detect when, in this case the pujoljulia.com domain, sends a message to analyticstestlab.com). Notice as well that when the event listener is dispatched, the handleRequest(); callback is called.
  • handleRequest(); callback parses (converts the string to the object) the message in order to understand what information has been sent. Realized that depends on the ‘type’ parameter sent by one of the Data Layer methods (setItem(), getItem() or deleteItem()) the code will set, get or delete the items accordingly.
  • Last think to know is that if the ‘name’ parameter of getItem() method is ‘all’, an object with all items stored will be returned. Please take in account that for the good performance of the methods, all items (regardless to be sessionStorage or itemStorage) have to have a different name.

Now what you have to do is copy this code and paste it to the site you want to be your ‘main site’ since all items are going to be saved at this domain. Remember that is really important to add in the ‘whitelist’ variable just those domains you want them to be able to set up cross domain items.

5. The ruler: Tag management system

The Tag Management System is the last part of the puzzle, as it is the tool that helps us to manage, organizes and dispatch all data collected. Let’s take back the car configurator example we’ve discussed earlier to show a real use case and this time let’s use GTM for it. Imagine that below buttons are features a user can select in the car configurator:

Feature 1 Feature 2 Feature 3 Feature 4

Let’s say the brand has realized by analyzing the figures collected by GA/AA that the users whose add consecutively Feature 2 then Feature 4 and Feature 1 are keen to ask for a Test Drive, so let’s make it easy for them and show a popup offering the test. Please click the blue button, then the orange and finally the green one. Voila! Take a look at the ‘Resources’  ‘Application’ tab of the console’:

popup-item-storage

Display a pop up it’s just an example, what is really important is that you have the information available for every domain in your site and ready to play around with that data through GTM, so for example if a user clicks on the pop up and goes to Test Drive’s page but eventually does not apply, you can save this data to Floodlight/AdWords or whatever DMP pixel and retarget them to fulfill the Test Drive form. It’s not amazing?

Here are the steps on GTM to make it possible:

1. dataLayer.push()

First step is to know when a user presses any of the above buttons and which one, for that reason if you work with GTM ask the website developers to place the corespondent dataLayer.push()  calls for each button (you can do it with jQuery but you already know that this is dangerous!):

data-layer-push-gtm

Notice that in my case instead of dataLayer.push() I’ve used dataLayerGA.push(), since as my Data Layer’s object is already called ‘dataLayer’. This change can be rapidly made by modifying it on GTM snippet.

2. Trigger

GTM-trigger

3. Custom variable

GTM-variable

4. HTML Tag

And finally we’ve to build an HTML tag that will dynamically construct the array inside the Item Storge and trigger the pop up when the business rule Feature2:Feature4:Feature1 be accomplished.

GTM-tag

And here is the code:

A simple piece of code that checks if the item exist, if so append new values otherwise create the item. At the same time is checking if the business rules is complete, if so triggers the pop up.

Final thoughts

I hope this post inspire you to use item storage with many different creative ways and I also hope that the domains don’t be a hinder for you. As usual I will leave in this section some things to be aware before you leverage on my code:

  • The code works across browsers, tested on Chrome, Mozilla, IE, Opera and Safari. The problem with the last one is that the information is deleted when the browser is completely closed besides the browser itself shows that the data has been saved on localStorage.
  • Be mindful that each time you use one of the cross domain item storage methods, you’re making postMessage() requests between different domains, so don’t abuse of them and use the default browser methods (sessionStorage.setItem()/localStorage.setItem()…) when you are at the ‘main domain’.
  • That feature can be used for advertiser purposes like I did with an image pixel by saving an ID cookie, here it’s basically the same concept but saving an ID Item.
  • The same process could be done through DTM or whatever other TMS, the concept of Data Layer I’ve been explaining to you is tool agnostic, it just provides methods to improve your tracking strategies.
  • Download the entire Data Layer code from here. Notice that it contains the four methods discussed as well as some variables at the top (inside digitalData object) to manage some stuff cross methods. Copy and paste it to your site with the discussed amendments.

I’ll be happy to hear your comments or possible improvements. Keep calm and keep tracking!

2 Comments

  1. Great post! Thanks for sharing all this. I’m already using session storage, to reduce the quantity of cookies used in our analytics, but I hadn’t appreciated the cross-domain potential, so I’m saving this info for future use. – Thank you!

  2. Hi Alison,

    I’m very glad you find it interesting and useful. Definitely as analytics implementers and since the growing demand of first party data tracking, we all should leverage more on item storage instead of cookies to take advantage of its store size capacities.

Your comment

Your email address will not be published. Required fields are marked *