Tuesday, October 06, 2009
RSS Metrics: The Good, the Bad and the Invisible
Metrics matter - it's rule #1 for marketing programs. If you can't measure the success of a program (whatever success means to you) then you have no idea how well it's working and whether you should invest in it, tune it, or kill it. If you can't measure it, you can't manage it. Simple, really. So online marketers invest a lot of time setting up measurement systems for their online programs, such as Google Analytics for web sites and open / click through tracking for email marketing programs. It's even possible to get stats-addicted, endlessly micromanaging and tuning online programs while the bigger picture - and the bigger opportunities - pass you by. But that aside, getting to know your basic metrics - and their trends - is fundamental. That's true for bloggers and new media sites, because we all feel good when our stats go up and feel in our guts when a key metric burps. So how come the state of RSS metrics is so parlous? Most bloggers focus on their RSS circulation as the key metric, and the only other useful metric commonly available is reach (more on both these below). RSS services like FeedBlitz and FeedBurner give you that top line circulation number, even though it's almost certainly meaningless. RSS Metrics: The BadScreeeeech. Rewind. Circulation is "meaningless"? Yup. Not because the number isn't accurate. It's certainly the best number the relevant service can come up with, and it can be tuned or altered from time to time to "improve" it. No, it's not because it's inaccurate (but, man, we can surely debate that until the proverbial cows come home). No, circulation is mostly rot because it's not a particularly useful metric to be tracking. It's analogous to the total number of email subscribers in your email list. As any decent email marketer will (or should) tell you, size doesn't matter; it's quality that counts. And quality is largely measured by metrics such as open and click through rates. In other words, it's how many recipients interact with your mailing that determine how successful each mailing is. Same with advertising - better targeting yields better response rates which, in turn, command higher prices. And so it should be with feeds. RSS Metrics: The GoodSo where are we in RSS-land? We have subscribers (or as we call it, circulation), the total (-ish) number of subscribers to your RSS feed, and then there is reach, which is a measure of how many of your subscribers have interacted with your feed. Reach is a good quality metric, as it tells you how much activity your feed is generating. While reach will vary from day to day and post to post, the trend in reach will tell you whether you are gaining or losing attention from your readership. Reach, not circulation, is what you should really care about. The FeedBlitz RSS service calculates your reach on any given day by determining how many unique readers interacted with your feed (either opened it or clicked through). Useful, yes. But sadly reach, too, has its problems. Your reach total is probably under-reporting activity, because it has the same problems as email open rates: If the subscriber doesn't have images displayed you can't track the act of opening the email or reading the RSS article, because it's the call to the server for a tracking image that is counted. No images, no counting. Instead, for that subscriber, you have to catch them when (or, rather, if) they click through. RSS Metrics: The InvisibleHere's where the bad news kicks in. Did you know that you are simply missing out on almost all subscriber clicks on your RSS feed? That you're not tracking the vast majority of these subscriber interactions? It's true. Unless our invisible subscriber above happens to click through to the source article (usually by clicking on the post's title) - if they click on anything else - dollars to doughnuts you're going to miss them. And that means you're not getting the big picture at all. In fact, perhaps (or even probably) you're getting only a very small fraction of it. Why? Because you're not counting clicks on links that are within the post. If our invisible subscriber clicks on anything inside the post - anything at all - they won't be tracked and that subscriber's activity missed. By way of example I offer you TechCrunch's RSS feed at http://feeds.feedburner.com/techcrunch (not to pick on TechCrunch, by the way; I'm just using a well-known technology feed to make my point). TechCrunch, like most new media companies, liberally sprinkles its articles with links, some internal to the site and some to the third party sites mentioned in the post. Looking at the current feed I see the article R.I.P. Good Times: One Year Later - which has sixteen (16) links in the post, not including ads and FeedFlares (remember, you're accessing the posts via the feed http://feeds.feedburner.com/techcrunch not their website). TechCrunch has link tracking enabled via FeedBurner, and so the link back to the site via the article's title is tracked. You can tell because the link doesn't look like a TechCrunch site link; it has codes and funny characters (a tilde: ~ ) in it if you hover over it with your cursor / mouse pointer. Not so those 16 links inside the post. They're regular URLs - no tracking. Now, let's say you're one of the bajillion people who subscribe to TechCrunch in your RSS aggregator. What do you click on? Do you click on the post itself, only to read exactly the same article online (only with more ads)? Nah. More than likely you click on the links inside the article and head off to read about Sequoia Capital or the the deadpool or the crunchbase or whatever else is linked to in the post. Which means that when you do that, the RSS feed metrics will under-record the feed's reach because your click isn't counted. The good folks at TC simply can't tell you how many of their RSS readers click on links inside any post because they're simply not collecting that data. And nor are you for your blog. If you extensively use links in your posts you're absolutely, positively, no-doubt-about-it missing out on feed-based activity. If the likelihood is that these are, in fact, the links that are being clicked on by subscribers, can you imagine how much intelligence, how much useful information is being lost? Does that translate into lost revenue? Remember, if you can't measure it, you can't manage it. You're optimizing your online programs with massively incomplete information, and (worse) you don't even know how incomplete your knowledge is. RSS Metrics @ FeedBlitz: Making the Invisible VisibleWow, talk about burying the lead. Anyway, as of now, the FeedBlitz RSS service ALSO tracks internal links inside feed posts automatically. These links will appear on the RSS report and also make the reach figures larger (because we'll be capturing more activity) and more accurate (for the same reason). How much larger will depend on how often you use links inside your posts; the more often you add links within a post (I think TechCrunch, with double-figure counts, is fairly extreme) the better the metrics and the more likely you are to see your reach rise as a result. As an example, read this article at http://feeds.feedblitz.com/feedblitz (why not subscribe while you're there!). Look at the links inside. Coded. Tracked. This will be the first article - possibly ever - on an RSS feed where I, the feed owner, will be able to see all the activity the post generates. Finally, fret not, SEO mavens: all the links are 301 redirects, giving you the full Google-juice benefit. Better yet, if you use your FeedBlitz feed to power your email marketing, you get the same benefits too. This is a first for online social media marketing. Cool. Marketing metrics matter. Get the whole story and start a trial today. Labels: features, reporting, RSS
Thursday, October 01, 2009
FeedBlitz RSS Traffic up 17x over last week
If you were affected by the ho-hum availability of the FeedBlitz RSS service recently (we're A-OK now, by the way), here's why: Traffic jumped by a factor of 17 over this time last week and it took us too long to get to grips with it, for which I apologize. I hope this post will help explain why that happened, what we did and didn't do, and what we have learned from the experience. ChronologyLate last Friday (September 25 th) we noticed that incoming traffic had - very suddenly - started to clog the RSS service. We dug around a little, found out why, and added a new server. That seemed to handle things nicely and that was that - or so we thought at the time. Over the weekend the service was doing OK - a couple of flags here and there but nothing that seemed to merit urgent attention - and this continued through into Monday. So far, so good, and I was able to talk about the new integrated comment feature instead. Swell! Come Tuesday morning, though, and things weren't so happy. So we span up yet more servers to hold the fort (let's hear it for Amazon EC2 and cloudy services!), which seemed to settle things down. Yesterday (Wednesday), however, the traffic was again swamping the virtual server array and it was clear that simply spinning up more of the servers we had been using wasn't cutting it. So, instead, yesterday morning we hauled out a series of much larger servers into (from?) the cloud. At one point we had 7x the number of servers running than we did a week ago. Right now, the computing capacity currently deployed to serve feeds is more like 30x more than this time last week. ResultsThe good news is that the current infrastructure is now holding its own really well - in fact, we're starting to take some of the servers down now that we understand better the nature of this flood and how to handle it. There's plenty of headroom left right now to cope with at least a doubling of traffic should that happen and RSS delivery is currently extremely stable and responsive. From a numbers perspective, comparing yesterday to a week ago, RSS traffic we served went from just under 15 requests per second to 251 requests per second yesterday - that's the 17-fold increase. That's a huge change in what had otherwise been a predictable service with predictable traffic patterns. Questions, Questions.So that's what we did - but where did all this traffic come from? And how come we were so surprised? Let me take the second question first. After about 6 months running the RSS offering we know (or thought we knew) how it worked and how it changed. Basically, as we incrementally acquire new customers for the service, the load on the service grows incrementally in parallel. Most of the feeds we see have circulations in the thousands and tens of thousands, nothing scary about that at all and well within the ability of last week's infrastructure to handle for the foreseeable future. We had no idea last Friday that this was about to change. At the end of last week a new client started to use FeedBlitz to serve their feed. Their feed was not being accessed via traditional aggregators, though. Instead, it was being accessed by an automatically updating browser toolbar. With a circulation, apparently, of around 7 million individual users. So when a user with the toolbar fires up their browser they fetch the feed (and because the toolbar itself isn't particularly well-written, they keep on asking for the full feed, despite the headers we send back. Grrr!). Literally one minute things were fine, and the next, overload central. The toolbar factor is important. By way of background, most bloggers and content publishers have large portions of their audiences consolidated by web aggregators, such as Google Reader, email services like FeedBlitz, or have well-behaved desktop aggregators that know not to keep hitting up feed every 10 seconds. So even if a blog network comes to us with an audience of, say, a million RSS subscribers, it's more than likely that a good half of those will be handled by a few large aggregators, and the load from the personal systems might add a million hits or so a day to the system or so. That's easily handled. RSS aggregators are well behaved too; they understand things like entity tags in HTTP headers which help minimize traffic. The toolbar added an extra 20 million hits per day to the infrastructure. It's therefore roughly equivalent to the load generated by a blog with a multi-million subscriber RSS feed. Does anyone in the real world have a feed that size, anyway? I don't think even TechCrunch's or Scoble's RSS audiences are that big. So, no, we just didn't expect this kind of load appearing, unannounced, overnight. Looking AheadSo, anyway, hence the performance issues earlier this week. We strive to do our best here but for a while we weren't able to keep up. That's personally disappointing for me and I am certainly sorry for the inconvenience wrought on some of our clients as a result. The good news is that the RSS infrastructure is now not only better able to cope with a similar sized surge in the future, we're better able to ramp it up quickly and aggressively as and when we have to. One key lesson learned is that we scaled too conservatively when the initial hits came in, causing us to have to scramble later on - weekend traffic is always lighter than weekday and we failed to take that into account adequately. With hindsight we should have overscaled at first, then pulled back as the full traffic profile became clearer. We won't be making that mistake again. Geekery: AWS, EC2, S3 and MemcachedTechnically, there were a few surprises as well as we drilled into how to improve performance. You may not realize this, but when we serve a feed to you - say http://feeds.feedblitz.com/feedblitz - we're not simply sending a static file. The feed is subtly different for each individual user. Which is why we need beefy servers to handle the load because FeedBlitz RSS is CPU-limited, not bandwidth limited. In other words, serving a feed is work for the computer involved; we're not simply yanking the file out of a local cache and declaring victory. But talking of caches, here's another interesting technical factoid we discovered. Like many services, we use a technology called memcached to help scale. We'd assumed that it would work well in the EC2 cloud because there was no reason to think otherwise. And we were wrong. Our instrumentation in the middle of this episode clearly showed that caching and retrieving data from Amazon's S3 service was consistently an order of magnitude faster for servers under stress than fetching the same data from an EC2-hosted memcached server. My assumption is that there are environmental optimizations within the AWS environment that grease the electronic skids for S3 requests. If you're using memcached inside EC2 my recommendation is that you instrument the code you're using - memcached might not be working as effectively for you as you think it is. For what it's worth, we had been using memcached as a primary cache with S3 as a backstop. Now we just go directly to S3 instead. It's way faster. The Bottom LineSo there's the scoop. Big flood of traffic, didn't scale fast enough, but we're good now. Post ScriptOh, and by the way: If you're planning on dropping a few million subscribers on us overnight, please do. We can handle it now. Just give us a heads up first, okay? Thanks!
Monday, September 28, 2009
Why Separate RSS Feeds and Comments?
This has been bugging me the last few days as we've quietly extended support for commenting within FeedBlitz (FYI comment links now appear for Blogger blogs whereas they didn't before, for example, and new comment icons appear in FeedBlitz RSS feeds where appropriate). But it kept on striking me as odd - why do feeds force subscribers to subscribe to something else (a site comments feed or an individual post's comment feed), or to visit an alternative page or site to see the current conversation? Why aren't the latest comments, you know, just there, in the feed, when it's read? Why are we making readers to go to all this extra effort? Surely the conversation is part of the content, and it surely makes sense to have the conversation in the same context as the content, no matter where it's being consumed. From being bugged to taking action. As of now, all new feeds created on the FeedBlitz RSS service will include the 5 most recent comments if we can find them. You can change that (from none to the ten most recent), along with the entry title (such as "Talk to me!" instead of the default "Comments" and the text that displays when there are more comments available than your settings permit to be shown. Go to RSS / Settings and expand the Per-Post Customization area to access the comment integration settings. Publishers with existing feeds will can enable the feature at the same place. The goal here is to increase engagement and user attention by providing the dialog in the context of the feed. And to see it in action (I have a limit of 5 comments set up and have changed the default text to "Join the Conversation!") add http://feeds.feedblitz.com/feedblitz to your aggregator or RSS reader. The one gotcha is that if you're using a third party comment service (such as Disqus) then it's unlikely that the relevant tags are going to be in your feed so you may not get this feature to show up. Yet. Anyway - do chime in here, please. Corruption of the purity of the feed ... or an interesting idea? Join the conversation! Labels: comments, FeedBlitz
Friday, September 25, 2009
Maintenance - Saturday Sep 26 2009
We'll be running some maintenance on FeedBlitz late Saturday evening, US Eastern. We aim to keep any disruption to a minimum during the process.
Thursday, September 24, 2009
Accessibility
 FeedBlitz has added an audio version of the visual CAPTCHA (which has also been slightly updated) used on all FeedBlitz subscription forms. The API has also been extended to add a new audio tag to the captcha XML.
Tuesday, September 22, 2009
Dynamically Branding User-Defined Newsletters
FeedBlitz has long had a very easy way for marketers to enable dynamic user-defined newsletters using the tracking API call: http://www.feedblitz.com/f?Track=<user-defined-url>&publisher=<publisher-id>where you substitute in the appropriate values (your <publisher-ID> is your subscriber ID, visible at My Account / My Profile). It's great for users who use search, for example, which generates an RSS feed under the covers from which you can then generate a nifty, personalized email newsletter. When the email is sent it's branded with the publisher's "master feed" - enabled in the advanced template editor. This works great for when you have a single graphic design that can be consistently applied across all your newsletters. What we're finding here at FeedBlitz is that as we grow, the complexity and requirements of our larger customers are changing and growing too. We're finding that new media publishers often have multiple brands within their portfolio, and that they want to brand a user-defined email from site X with site X's branding - but want to define user-defined newsletters from brand Y with brand Y's branding. Master feeds don't cut it here, since they're a one-size-fits-all solution. So, for new media, multi-branded sites, we've added a new (optional) parameter to the tracking URL - portal. The portal is the list ID of the newsletter whose branding you want to use for the user-generated emails (newsletter list IDs are visible at Newsletters / Settings / Content Settings). So using the above example, this: http://www.feedblitz.com/f?Track=<user-defined-url>&publisher=<publisher-id>&portal=Xgives emails a different branding from this: http://www.feedblitz.com/f?Track=<user-defined-url>&publisher=<publisher-id>&portal=YAs long as there is a template defined for "X" and "Y" everything works. There are a couple of restrictions: once a given user-defined URL has been associated with another feed's template in your account, the only way to change it is by using a template editor. Secondly, you have to own the list you're using to brand with, for pretty obvious reasons. Now, start to imagine the possibilities when you combine this feature with the multi-feed templates discussed yesterday...
Monday, September 21, 2009
Styling Sizzle with Multi-Feed Newsletters
Since its inception, FeedBlitz has been automatically converting single RSS feeds into formatted, indexed, tracked email marketing newsletters. There are services - such as FeedBlitz's RSS service and Yahoo Pipes - that allow multiple feeds to be merged, blended and spliced into one, so it's also easy to munge multiple feeds into a single newsletter. But the problem with splicing is that it merges and mixes all the source feeds into one unified feed. While that's fine for many applications, what if you want to have a single newsletter, structured with distinct sections? Or what if you want to build a mailing with email widgets in the sidebar, such as special offers, or your most recent tweets, or your last.fm playlist, or your latest photos or videos, all without having to manually edit the template all the time? You can't do that with a single feed - whether spliced or standalone - and simple templates. Or, rather, you couldn't ... until now. One Newsletter, Many Feeds, No Extra Work.Because - surprise! - now you can. Power users can use the advanced template editor to include multiple feeds in a single mailing, producing significantly more complex and visually appealing (yet still fully automated) marketing communications. Well, ok, there's a little extra work, but only what's needed to re-jig your template. Once that's done, it's all automatic from then on. How It Works and How to TestThe magic is done by the inclusion of a new custom tag in the advanced template editor, called <$Feed$>. I'll run through some examples in this post, and so if you want to play along you'll need an advanced template to experiment with. If you don't currently have an advanced template, a great way to create one is to use the easy template editor to create a basic layout, and then flip into the advanced editor so you can roll up your sleeves and get your hands dirty with the tags. If you want to test advanced templates safely without affecting production mailings you can clone your existing newsletter at Newsletters / Settings / Clone Newsletter. Cloning copies everything except subscribers, so you can experiment safely. Once you're done you can simply copy and paste the advanced template you've built back into your main list. The <$Feed$> tag is briefly documented in the legend area under the advanced editor; the advanced editor itself is available at Newsletters / Settings / Advanced Email Template Editor. Adding Multiple Feeds to your TemplateSay you want to build a newsletter with multiple sections, each section sourced from a distinct feed. For example, you might have a classroom newsletter built from the teacher's blog, but you might want to include the latest from the PTA and the principal as well. Or you're a realtor, and you want to include an industry news feed or the latest from your firm's mortgage broker in your mailings. The new tag helps you set this all up. Before you start, you will need to know the RSS feed for the other content sources you want to add in. Assuming you want your main content up first, you need to find your last <$BlogPosting$> tag. Place the cursor where you want the new content to go and type in:
<$Feed="http://foo.example.com/feed"$>
Obviously, replace the dummy URL above with the real URL you want to include. This tag starts the inclusion and formatting of your additional content. Now, after that, type in just:
<$Feed$> This tells the FeedBlitz template engine where to stop including the external content. You need both; if you forget the ending tag things get chaotic fairly quickly! Now all you have to do is, between these two new items, insert standard template text, formatting FeedBlitz template tags etc. Basically, you're creating a template within a template, so you can create a title, add graphics, and if you want to add post content (which you do, otherwise you wouldn't be doing this), you'll need to have a pair of <$BlogPosting$> tags inside the feed tags we just added. If you want the layout mirror your current mailings, you can simply copy and paste. Use the preview to see how it all looks as you wrap up. You can repeat the process as much as you like (although I think it's better to limit includes to just two or three additional feeds, tops - if you find you're including more you might be better off offering multiple newsletters instead). What HappensThe <$Feed$> tags insert the content and then format it according to the code inside them, just like the main template. Used as described above, they will only insert content if there's something in the feed that matches the main newsletter's criteria - so if the class teacher has 2 new articles this week but the school Principal didn't write anything, the Principal's subfeed won't be included in the mailing. By default, posts will be formatted according to the same rules as the main Newsletter, so posts will be truncated and tracked consistently. The main feed - the one defined as the article source in Newsletters / Settings / Content Settings / The Basics - always defines when a new mailing should go out, no matter what changes in the included extra feeds. If you want the Principal's mailings to go out even when the teacher hasn't written new content, you should either use a separate newsletter or a newsflash mailing. By default, this means that elements inside a <$Feed$> block will not appear on landing pages such as subscription forms. <$Feed$> ParametersYou can tune the behavior of the included third party feed in your mailing with a couple of optional parameters. Changing the amount of text to displaySay in our hypothetical classroom newsletter that we are including all the teacher's post content in the mailing, but for the sake of brevity we want to limit the included content to just the initial sentences - if folks really want to read all the Principal's musings they'll have to click through. By default, included feeds are formatted the same way as the main content, so we have to change that. It's easy: Simply add the "maxchars" parameter to the opening <$Feed$> tag and the post will be limited to that number of characters. FeedBlitz will truncate intelligently at the 350 character mark, backtracking to the nearest white space and preserving formatting and links up to the cut off. I suggest 350 as a good limit to use since 350 typically fetches the first two or three sentences. With this limit in place your opening tag would look like this:
<$Feed="http://foo.example.com/feed" maxchars=350$> You can use the maxchars parameter to include more than the main newsletter if you want, just by making the value big enough (e.g. 100000). Remember, you only change the opening tag. The ending tag (required for each starting tag) is always simply <$Feed$>. Turn your <$Feed$> into an email widget!The other parameter you can use is the "count" parameter. You can use it to force the included feed to display the most recent N articles, regardless of their date or time. So this:
<$Feed="http://foo.example.com/feed" count=5$> always includes the most recent 5 articles, no matter what. A side effect of this is that feeds with "count" specified will appear on landing pages, which is just what you want for a sidebar widget. Using the advanced template editor you can add an external feed to the end of the main content - or you can create a sidebar and have your recent tweets, playlists, or comments show up automatically there. If you only include the blog post's title between the included feed's <$BlogPosting$> tag then you have a little headline widget going on. Just remember that the widgets you create are static - they are built when the mail is sent, not when it is read. Examples!The following examples show how to use "FeedBlitz News" as a third party feed inside your advanced template.
- Basic post content
<$Feed=http://feeds.feedblitz.com/feedblitz$> <$BlogTitle$> <$BlogPosting$> <$BlogItemTitle$> <$BlogItemBody$> <$BlogPosting$> <$Feed$>
- Truncated text
<$Feed=http://feeds.feedblitz.com/feedblitz maxchars=350$> <$BlogTitle$> <$BlogPosting$> <$BlogItemTitle$> <$BlogItemBody$> Click to read more... <$BlogPosting$> <$Feed$>
- Recent Headlines Widget
<$Feed=http://feeds.feedblitz.com/feedblitz count=5$> <$BlogTitle$> <$BlogPosting$> <$BlogPosting$> <$Feed$>
One Newsletter, Many FeedsThe great benefit of this new tag is that it allows you to break the "single column" layout typical of many automated newsletters without creating any additional work for you on a regular basis. Just tweak the template and you're up and running, add more content blocks and sidebars, and there's now no reason why your automated newsletter can't be as rich and compelling as any hand built one. It's just that, unlike other newsletter contact services, each mailing takes you zero minutes to create, so you can focus on your mission. FeedBlitz is the only RSS to email service that allows you this flexibility and power. Try it out today at Newsletters / Settings / Advanced Email Template Editor.
Labels: email templates, features, FeedBlitz
Friday, September 18, 2009
Measuring Email Marketing Metrics
The FeedBlitz delivery metrics report now includes percentages as well as raw numbers for your opens, forwards, bounces, unsubscribes etc. The changes are part of a general effort to give publishers greater insight into their newsletters' performance. The latest changes also allow you click a given schedule class to easily differentiate between overnight mailings and those sent to Twitter and IM subscribers which show up as "Express." Don't forget that, as you view the report, you can click on the chart's graphic elements to drill down further into the data. The delivery metrics report is at Newsletters / Reports / Delivery Metrics report on the FeedBlitz web site. Labels: features, FeedBlitz, reporting
Wednesday, September 16, 2009
Delays Weds Sept 16
Overnight issues have delayed mailings; we're picking up the pace now. Sorry for any inconvenience. Update - we backfilled completely by the afternoon; services were fully functional early morning. We're reviewing what happened overnight to increase the service's robustness to the class of failure that occurred.
Friday, August 28, 2009
Automatic Category Feeds for Everyone!
|
© FeedBlitz - Blog and RSS Email Solutions | www.feedblitz.com | info@feedblitz.com | Privacy | Terms of Service
|