Tag Archives: web log analyzers

Version 3 Sneak Peek #2: Log files caching

Leave a reply

 

Part #1: Redesigned reports and 30% discount
Part #3: Beta available and Hybrid analysis

On several occasions in the past, users have asked why we don’t import data from log files into some kind of database. The answer is still the same: using any kind of database actually significantly reduces performance, even with some of fast libraries like SQLite. Sure, it would be much easier to develop, but the trade-off on the users’ end would be enormous, if we want to follow the original idea of web log analyzer capable for free on-the-fly filtering and creating ad-hoc reports.

So Web Log Storming utilizes series of our own algorithms to:

  • Parse and analyze text log files as fast as possible
  • Organize data in complex model, so it doesn’t take any more memory than it’s necessary
  • Provide fast routines to search and filter reports “on-the-fly” by any available data

 

It is clear that any of third-party libraries, database or other, cannot be more optimized for specific tasks that Web Log Storming requires, then those developed by ourselves from the scratch.

Still, starting with version 3, we have introduced a small improvement with optional log files caching. Basically, Web Log Storming will now analyze text log files, convert to numbers, dates, IP addresses, etc, and then the new step: save parsed data to cache files in format suitable for further usage. Next time when you analyze same log file, software will check if cache exists and read a prepared cache, instead of parsing text file again.

Log files cache options

Naturally, in case caching is enabled, first reading will be slower for the exact time needed to save data to the disk, but each next reading will be significantly faster for the time needed to convert texts to operable data. The speed change will depend on various factors, such is the typical size of individual log files and your preference if you want to keep cache compressed or not, etc. According to our tests, here’s a approximate rough speed comparison:

 

No caching  Uncompressed  Compressed
First analyzing 100% 100-120% 120-150%
Each future analyzing 100% 50-70% 70-90%
Combined first +4 future
 100% 60-80% 80-102%
Combined first +9 future 100% 55-75% 75-96%
Combined first +49 future
 100% 51-71% 71-91%
Additional space on the disk 0% 100% 10-15%

Typically, for moderate sized log files, you can expect a better performance (closer to lower ranges). For very small individual files (up to few kilobytes) or very large files (hundreds of megabytes or more), caching might not be such a good choice.

As you can see, best total speed improvement is with uncompressed cache, but at the cost of the additional disk space. By setting a caching options, you can decide what works best in your case.

Upgrade policy and “Early Bird” 30% discount

As our users probably already know, we switched to subscription upgrade model years ago. That’s why we don’t specifically charge for major updates like this one: update is free for anyone who bought or upgraded within at least a year before a release day or for anyone who bought a lifetime license.

So feel free to download and try current version and, if you decide to buy even today, you will get version 3 for free. As a special promotion, you can use this coupon for 30% discount:

WLS89445EARLYBIRD

The coupon applies to any purchase (new license or an upgrade) and will be valid until v3.0 is released.

* * *

Feel free to send us your thoughts regarding version 3, either by commenting here or by using a contact form. We’ll be happy to hear your opinion.

How to track AdWords campaign without using cookies

Leave a reply

People often think that it’s not possible to track successfulness of marketing campaigns without using cookies and/or JavaScript, which isn’t true. I will explain how it can be done with our Web Log Storming, but maybe the similar results could be accomplished with some other web log analyzer. Also, even though the example shows AdWords, you can track any other advertising results in a same way.

Setting up an AdWords campaign

The goal is to somehow differentiate visitors that came from different AdWords campaigns, groups and/or ads. It’s quite easy, actually, as only thing you need to do is to append some query text to Destination URL like this:

If you append “?anything” to URLs (including question mark, without quotes), this information will be included in server log files, while visitors will see the same content as if query is not there. For different campaigns and ad variations append slightly different query, for example:

  • https://www.mywebsite.com/?adw=test1
  • https://www.mywebsite.com/?adw=test2
  • …

Use it in Web Log Storming

After traffic starts coming in, Web Log Storming will be able to analyze this information and include it in reports. Easiest way to this is to enter “adw=*” into Parameters | File | Query field (note there is no question mark here). You can also use Lock button to keep this filter active as you switch reports.

Custom variables in Web Log Storming

While this parameter (filter) is active, any report that you select will be based on AdWords visitors only, making it easy for you to see how they perform.

Another tool that you might want to use is View | Based on IP only main menu option. If you select it, sessions from the same IP will be grouped as one, allowing you to examine it as a returning visitor.

Going further – using Goals

If you use Professional edition, you can also define Goals important for your business. Please read this article to learn how to do that.

With two Goals defined, this is what you’ll get if you choose Queries report:

Query Count … Goal1 % Goal2 %
adw=test2 68 … 45 66.18% 8 11.76%
adw=test1 27 … 3 11.11% 27 100%
…

As you can see, for each defined AdWords destination URL (campaign, group or ad), you get metrics that show how they perform against specific goals. In this example, campaign Test1 works perfectly for Goal2, while Test2 works better for Goal1.

How accurate is this?

As you might know, identifying visitors by IP might not be perfect. There is no guarantee that visitor will have same IP next time he visits website, and there is no guarantee that he’s the only one using this IP (proxies). However, I must argue that cookies are not perfect for this job either. For example, visitors can allow session cookies only, automatically clear cookies after closing a browser or block cookies and/or JavaScript completely.

Anyway, we are all probably sure that less people use proxies these days than few years ago, more people are on a broadband with fixed IP addresses and more people intentionally block cookies and analytics scripts. In other words, try this and you might be pleasantly surprised.

Web Log Storming: up to 40% competitive discount

Leave a reply

In addition to an educational 30% discount, we have just announced a competitive discount of 20% or 40% (depending on product). We believe that this could be a nice opportunity to either switch to Web Log Storming or to use it as an additional analytics tool. You just need to send us a screen shot with a proof – it could be a picture of the About box with your name in it (if it’s a desktop solution) or a picture of a web page for a hosted solution.

For paid packages (either desktop or hosted) the discount is 40%, which means that you can get new Web Log Storming license for only $113.40 (US).

And this could surprise you. The discount is currently available even if you use free analytics package, but with an additional condition: you must use it for at least two months (make sure you set date range accordingly when taking a screen shot). In this case discount is 20% and your price would be $151.20 (US).

The offer is also available for the upgrade from old version of Web Log Storming ($47.40 / $63.20).

Visit Web Log Storming website for more information

Which web log analyzer should I use?

1 Reply

Web log analysis softwareAnybody who maintains or owns a website should be interested in access statistics and can make a good use of web log analyzer. However, there are too much similar products on the market and users can be pretty much confused. They often think “Why should I pay for website stats software when there are free alternatives?” Read on to find out.

Regarding price, web stats software can be divided into three main groups:

  1. Free or Open Source
  2. Middle-priced
  3. High-priced

Having this in mind, it looks like users can be divided into three groups as well:

  1. Hobby
  2. Small and middle business
  3. Large corporations

Free analyzers

If you run a personal or hobby site that you don’t make money from, than you would probably be satisfied with free software: Analog, Webalizer, AWStats – to name a few. It’s natural, after all: you don’t depend on results of this software and you can’t do much to raise zero earnings. Free analyzers will satisfy your curiosity, and not much more than that.

There’s another “free” analyzer that’s most popular in last few years: Google Analytics. It actually falls somewhere in between first and second group and it deserves separate section of this article (see below).

Commercial analyzers

If you depend on your business (online or offline), you’ll need more that satisfying curiosity. You’ll need to know as much as it’s possible about your visitors: their behavior, technical data (browsers, operating systems), “dead ends” on your web site, etc. You will also need to calculate specific conversions. Conversion is the ratio between total number of visitors and visitors that took a wanted action (software download, mailing list subscriptions, orders, etc). Depending on this analysis, you will change your web site to achieve better comprehension, better wording effects, better conversion, and better earnings eventually. In this case, you should consider some commercial alternatives.

But not all commercial software is the same. A lot of them offer almost identical set of functionality as free ones, with little differences here and there. Some of them even look like exact clones of each other. It’s very hard to examine dozen of similar software. If first few of them don’t offer something new and interesting, you’ll probably give up eventually and turn back on free alternative.

One may think: “Of course they are similar! They all take same log files as input and they produce reports as output. In how many different ways you can do this?” If this is what you are thinking too, you may be surprised.

Google Analytics

Currently most popular web analytics solution and the synonym for web statistics for lot of people. Like WinZip is considered as synonym for archive software, but does that mean that those two are best on the market?

Google Analytics is a decent web statistics application, for the price. It definitely has better set of features than other free alternatives. However, it’s JavaScript based, which does bring several advantages (ability to get screen resolutions, connection speed reports, more exact visitor detection), but losing some important benefits of good old raw log files at the same time. For example:

  • You must change your web pages to include scripts
  • Scripts take additional time to load
  • You are limited to page stats, while file hits (exe, pdf, zip, …) are ignored
  • Because of this, bandwidth report doesn’t make sense
  • You cannot see how bots (spiders) are (mis)behaving
  • You don’t have any information about visitors with JavaScript disabled – like they don’t exist
  • You must wait for 24 hours until stats are refreshed
  • Stats and goals from the past are lost (from the period before including scripts)
  • And last, but not least, your stats are available to a third-party who can (and does) use it for self purposes. See the article “What price Google Analytics?” by Dave Collins.

While it offers number of cross-reports, you don’t have the full control and you can’t drill down to the level of individual visitor. You can’t identify the spiders that pull your entire website several times a day, wasting the bandwidth. There’s also no easy way to discover errors (“page not found”, “internal server errors”), hacker attacks, external pages that include images from your website or direct file hits (file downloads and other non-html documents). These are all serious drawbacks in our (and I believe everyone’s) business.

In essence, if you get over some of its disadvantages (at least regarding privacy), Google Analytics can be useful as an additional tool to log file based analyzer, but depend your business solely on it would be a major mistake.

Web Log Storming

Interactive web log analyzer - Web Log StormingWhen we started online business, naturally, we were in the same quest as you might be right now. By trying out free and commercial software at that time, we kept feeling that something is missing. Sure, you can see visits over the time, although some analyzers tend to mix up visitors and page views, which is unacceptable. You can also see top pages, referrers and browsers. But there was no easy way to drill down to the level of single visitor or hit, or to create custom cross-reports – easily limiting data by date or any other information available in raw log files.

That’s why we have decided to make an interactive web log analyzer that will allow users to extract the information that they need at that moment.

Web Log Storming is different. It doesn’t take the usual and fixed route: log file > analyze > report. It doesn’t make much unnecessary assumptions about what you need to know. It loads log files and lets you analyze them. It lets you look at stats from different angles, taking in consideration different parameters.

Note: Because of its nature, it’s intended for small and medium business use and it’s consequently priced (middle-range). If your website is visited by tens or hundreds of thousands every day, this software is unfortunately not for you, as interactive features are hard to accomplish with such traffic.

Let’s name few use-cases.

Specific page visit (or download) per referrer. What’s the use of high traffic from Google if only small number of visitors actually goes further than quick and probably worthless glance over landing page? None. And sometimes you actually pay for those visits. With Web Log Storming you can limit referrer to Google (or AdWords campaign) and see any other report based on it (including page hits, paths, countries, etc).

List sessions with all details from a specific domain. Want to learn about your specific visitor – possibly big client? Do you want to know his first visit, original referrer, visited pages and time that he spent on each of them? You can draw a conclusion about his seriousness and interests and act accordingly. For example: you can discover that possibly important business proposal originates from 10th page of the rather generic Google search – it speaks a lot about how serious the proposal is.

See referrer report based on country. If your business rarely benefits from specific countries and you still have traffic from them, can you see which website they originate from? Is it a shady website that actually hurts your business and bandwidth?

Web Log Storming gives you this and much other exotic info, as we like to say, “on-the-fly”. Not much other analyzers can do that.

Add Web Log Storming to your “software to evaluate” list – take a free trial. And when you experience its possibilities, with your log files under the hood and your goals on your mind, you can decide what to do next. In the meantime, feel free to contact us for any questions or problems that you might have.