Tag Archives: linkedin

Web Log Storming: up to 40% competitive discount

10 reasons why web log analyzers are better than JavaScript based analytics

In this article we are going to point out some objective strengths of web server log analysis compared to JavaScript based statistics, such is Google Analytics. Depending on your preferences and type of the website, you might find some or all of these arguments applicable or not. In any case, everyone should be at least aware of differences in order to make a right decision.

1. You don’t need to edit HTML code to include scripts

Depending on how your website is organized, this could be a major tasks, especially if it contains lot of static HTML pages. Adding script code to all of them will surely take time. If your website is based on some content management system with centralized design template, you’ll still need to be careful not to forget adding code to any additional custom pages outside this CMS.

2. Scripts take additional time to load

Regardless of what Google Analytics officials say, actual experiences prove otherwise. Scripts are scripts and they must take some time to load. If external file is located on a third-party server (as it’s the case with Google Analytics), the slowdown is even more noticeable, because visitor’s browser must resolve another domain.

As a solution they suggest putting inclusion code at the end of the page. Indeed, in that case it would appear that page is loaded more quickly, but the truth is that there’s a good chance that visitor will click another link before script is executed. As a result, you won’t see these hits in stats and they are lost forever.

3. If website exists, log files exist too

With JavaScript analytics, stats are available only for periods when code was included. If you forget to put code on some pages, the opportunity is forever lost. Similarly, if you decide to start collecting stats today, you’ll never be able to see stats from yesterday or before. Same applies to goals: metrics are available only after you decide to track them. With some log analyzers, you can freely add more goals anytime and still be able to analyze them based on log files from the past.

4. Server log files contain hits to all files, not just pages

By using solely JavaScript based analytics, you don’t have any information about hits to images, XML files, flash (SWF), programs (EXE), archives (ZIP, GZ), etc. Although you could consider these hits irrelevant, they are not for most webmasters. Even if you don’t usually maintain other types of files, you must have some images on your website, which could be linked from external websites without you knowing anything about it.

5. You can investigate and control bandwidth usage

Although you might not be aware of it, most hosting providers limit bandwidth usage and usually base their pricing on it. Bandwidth usage costs them and, naturally, it most probably costs you as well. You would be surprised how much domains (usually from third-world countries) poll your whole website on a regular basis, possibly wasting gigabytes of your bandwidth every day. If you could identify these domains, you could easily block their traffic.

6. Bots (spiders) are excluded from JavaScript based analytics

Similar as previous point, some (bogus) spiders misbehave and they are wasting your bandwidth, while you don’t have any benefit from them. In addition, server logs also contain information about visits from legitimate bots, such are Google or Yahoo. By using solely JavaScript based analytics you have no idea how often they come and which pages they visit.

7. Log files record all traffic, even if JavaScript is disabled

Certain percentage of users choose to turn off JavaScript, and some of them use browsers that don’t support it at all. These visits can’t be identified by JavaScript based analytics.

8. You can find out about hacker attacks

Hackers could attack your website with various methods, but neither of them would be recorded by JavaScript analytics. As every access to your web server is contained in log files, you are able to identify them and save yourself from damage (by blacklisting their domains or closing security holes on your website).

9. Log files contain error information

Without them, in general case, you don’t have any information about errors and status codes (such are Page not found, Internal server error, Forbidden, etc.). Without it, you are missing possible technical problems with your website that lower overall visitor’s perception of its quality. Moreover, any attempt to access forbidden areas of your website can be easily identified.

10. By using log file analyzer, you don’t give away your business data

And last but not least, your stats are not available to a third-party who can use them at their convenience. Google has bought all rights for, at that time, popular and quite expensive web statistics product (Urchin), repackaged it, and then allowed to anyone to use it for free. The question is: why? They surely get something in return, as Google Analytics license agreement allows them to use your information for their purposes, and even to share it with others if you choose to participate in sharing program.

What could they possibly use? Just to give few obvious ideas: tweaking AdWords minimum bids, deciding how to prioritize ads, improving their services (and profits) – all based on traffic data collected from you and others.

Busting the Google Analytics Mythbuster

3 Replies

In the recent article at Google Analytics Blog author tries to bust several myths circulating in the public. You can find few half-truths and (intentional?) deceptions there that I simply can’t ignore.

As I mentioned earlier, Google Analytics could be a nice addition to the main analytics solution (even we are using it occasionally), providing that you don’t mind the baggage that comes with “free” label (What price Google Analytics?, Google Analytics – is it worth its price?, Google Analytics is not free, or search Google 🙂 for more). JavaScript based systems give some information that log files can’t, but they also suffer from several drawbacks that are limiting the value if used alone.

As each product has its own audience, I won’t question anyone’s decision to choose one type of solution or another, but some things simply must be said, regardless of how many people will read this compared to the original article. 🙂

MYTH 1: “You get what you pay for.” Google Analytics is free, which means the system is down a lot.”

I do agree that GA system is not down very often (if ever). Why it would be? They have more than enough resources to keep it alive, and imagine how much data they would lose in just one minute of downtime. But no matter how powerful their servers are your website will inevitably be slower. I doubt that you’ll find this particularly alarming, but still…

MYTH 4: Google Analytics is not really accurate

Google Analytics uses JavaScript tags to collect data. This industry-standard method […] discrepancies greater than 10%, it’s due to an installation issue. Common problems include JavaScript errors, redirects, untagged pages and slow client-side load times.

[…]

All web analytics tools face the same technical limitations posed by JavaScript tags […]

Ouch. This one is a main motivation for me to write the article. I’ll just comment phrases in bold (in the order of the appearance).

JavaScript tags are just one of methods used today. Even if we ignore custom in-house systems (based on whatever web developers use: PHP, ASP, Python, Ruby on Rails, …), pretending that still widely accepted server log file analysis don’t exist is at least an intentional delusion.
Expected discrepancies of 10% or below among JavaScript based analyzers could be true, but compared to log file analysis, they show 2 to 5 times less traffic.
It could be the installation issue only if visitors can be tracked with JavaScript. What about other traffic?
Again, we can talk about errors and slow connection only when JavaScript tracking is possible.
Saying that all web analytics tools face the same limitations is simply not true. JavaScript based web analytics tools do have these limitations, but not log file analyzers.

Pardon me if you don’t care about visitors that block JavaScript or click on a different link before tracking script is loaded, websites that directly link to images on your website, downloads of non-html files (PDF, ZIP, EXE, images, …), bandwidth usage, spiders, bots that pull down the whole website on a regular basis (wasting your bandwidth), direct access to scripts by hackers, etc, etc. Sure, with JavaScript analytics you can see trends and if you only care about marketing it could be good enough, but total number are not even close, and you can forget about other information that can be found in server log files.

MYTH 6: With Google Analytics you can’t control your data

Yes, you can control your data… at some degree. Google promises to resists the urge to analyze your data for own purposes (if you don’t forget to explicitly say so), but the fact is that they already have your data, right there. In this information era knowledge is a big asset. Sorry, but I don’t buy that they won’t ever “peek”, just a little. Probably under the excuse of “serving better search results” (or more likely, “serving better advertisements”). And I’m not talking about analytics only: they have search queries, e-mails, documents, appointments, instant messages, etc. They predicted Eurovision 2009 contest winner based on what people search and I should believe that they won’t silently use all the information they can for profits? Right…

Even if you do trust Google (and every its employee), you still can’t say that you fully control your data as it’s still on their servers. Anything can happen in the future. What if Google goes to bankruptcy? Okay, not likely, but possible. 🙂 Therefore, you can’t fully control your data, but don’t get me wrong: I admit that there are few pros. For example, you don’t need to think about backup – the data is much safer on Google servers than on your computer. 🙂

* * *

Disclaimer: the purpose of this article is not to persuade anyone to use server log analyzer instead of Google Analytics (I wrote another article for this 🙂 ), but to point out few things that are too easily overlooked these days, intentionally or not.

Which web log analyzer should I use?

1 Reply

Anybody who maintains or owns a website should be interested in access statistics and can make a good use of web log analyzer. However, there are too much similar products on the market and users can be pretty much confused. They often think “Why should I pay for website stats software when there are free alternatives?” Read on to find out.

Regarding price, web stats software can be divided into three main groups:

Free or Open Source
Middle-priced
High-priced

Having this in mind, it looks like users can be divided into three groups as well:

Hobby
Small and middle business
Large corporations

Free analyzers

If you run a personal or hobby site that you don’t make money from, than you would probably be satisfied with free software: Analog, Webalizer, AWStats – to name a few. It’s natural, after all: you don’t depend on results of this software and you can’t do much to raise zero earnings. Free analyzers will satisfy your curiosity, and not much more than that.

There’s another “free” analyzer that’s most popular in last few years: Google Analytics. It actually falls somewhere in between first and second group and it deserves separate section of this article (see below).

Commercial analyzers

If you depend on your business (online or offline), you’ll need more that satisfying curiosity. You’ll need to know as much as it’s possible about your visitors: their behavior, technical data (browsers, operating systems), “dead ends” on your web site, etc. You will also need to calculate specific conversions. Conversion is the ratio between total number of visitors and visitors that took a wanted action (software download, mailing list subscriptions, orders, etc). Depending on this analysis, you will change your web site to achieve better comprehension, better wording effects, better conversion, and better earnings eventually. In this case, you should consider some commercial alternatives.

But not all commercial software is the same. A lot of them offer almost identical set of functionality as free ones, with little differences here and there. Some of them even look like exact clones of each other. It’s very hard to examine dozen of similar software. If first few of them don’t offer something new and interesting, you’ll probably give up eventually and turn back on free alternative.

One may think: “Of course they are similar! They all take same log files as input and they produce reports as output. In how many different ways you can do this?” If this is what you are thinking too, you may be surprised.

Google Analytics

Currently most popular web analytics solution and the synonym for web statistics for lot of people. Like WinZip is considered as synonym for archive software, but does that mean that those two are best on the market?

Google Analytics is a decent web statistics application, for the price. It definitely has better set of features than other free alternatives. However, it’s JavaScript based, which does bring several advantages (ability to get screen resolutions, connection speed reports, more exact visitor detection), but losing some important benefits of good old raw log files at the same time. For example:

You must change your web pages to include scripts
Scripts take additional time to load
You are limited to page stats, while file hits (exe, pdf, zip, …) are ignored
Because of this, bandwidth report doesn’t make sense
You cannot see how bots (spiders) are (mis)behaving
You don’t have any information about visitors with JavaScript disabled – like they don’t exist
You must wait for 24 hours until stats are refreshed
Stats and goals from the past are lost (from the period before including scripts)
And last, but not least, your stats are available to a third-party who can (and does) use it for self purposes. See the article “What price Google Analytics?” by Dave Collins.

While it offers number of cross-reports, you don’t have the full control and you can’t drill down to the level of individual visitor. You can’t identify the spiders that pull your entire website several times a day, wasting the bandwidth. There’s also no easy way to discover errors (“page not found”, “internal server errors”), hacker attacks, external pages that include images from your website or direct file hits (file downloads and other non-html documents). These are all serious drawbacks in our (and I believe everyone’s) business.

In essence, if you get over some of its disadvantages (at least regarding privacy), Google Analytics can be useful as an additional tool to log file based analyzer, but depend your business solely on it would be a major mistake.

Web Log Storming

When we started online business, naturally, we were in the same quest as you might be right now. By trying out free and commercial software at that time, we kept feeling that something is missing. Sure, you can see visits over the time, although some analyzers tend to mix up visitors and page views, which is unacceptable. You can also see top pages, referrers and browsers. But there was no easy way to drill down to the level of single visitor or hit, or to create custom cross-reports – easily limiting data by date or any other information available in raw log files.

That’s why we have decided to make an interactive web log analyzer that will allow users to extract the information that they need at that moment.

Web Log Storming is different. It doesn’t take the usual and fixed route: log file > analyze > report. It doesn’t make much unnecessary assumptions about what you need to know. It loads log files and lets you analyze them. It lets you look at stats from different angles, taking in consideration different parameters.

Note: Because of its nature, it’s intended for small and medium business use and it’s consequently priced (middle-range). If your website is visited by tens or hundreds of thousands every day, this software is unfortunately not for you, as interactive features are hard to accomplish with such traffic.

Let’s name few use-cases.

Specific page visit (or download) per referrer. What’s the use of high traffic from Google if only small number of visitors actually goes further than quick and probably worthless glance over landing page? None. And sometimes you actually pay for those visits. With Web Log Storming you can limit referrer to Google (or AdWords campaign) and see any other report based on it (including page hits, paths, countries, etc).

List sessions with all details from a specific domain. Want to learn about your specific visitor – possibly big client? Do you want to know his first visit, original referrer, visited pages and time that he spent on each of them? You can draw a conclusion about his seriousness and interests and act accordingly. For example: you can discover that possibly important business proposal originates from 10^th page of the rather generic Google search – it speaks a lot about how serious the proposal is.

See referrer report based on country. If your business rarely benefits from specific countries and you still have traffic from them, can you see which website they originate from? Is it a shady website that actually hurts your business and bandwidth?

Web Log Storming gives you this and much other exotic info, as we like to say, “on-the-fly”. Not much other analyzers can do that.

Add Web Log Storming to your “software to evaluate” list – take a free trial. And when you experience its possibilities, with your log files under the hood and your goals on your mind, you can decide what to do next. In the meantime, feel free to contact us for any questions or problems that you might have.