Relax FAQ

Relax is a specialized web server log analysis tool for referrer information processing.

Relax reports tell you the quantity of visitor you are getting from the leading search engines, as well as the most popular keywords being used. This information is useful for improving search engine rankings or determining which keywords to buy from search engines.

Relax can be integrated with Etsin to, for example, check how different keywords are positioned in search engines.

Relax generates lists of URLs directing to your pages. This information can be used to tell who is linking to your site, and to trace obsolete or misspelled links.

All platforms where Perl is available should be OK. I have developed and tested Relax on SunOS, Windows 95 OSR2 and Windows 2000. All other platforms are untested and it's quite possible that some minor modifications are needed. Reports from BSD, IRIX and Amiga users indicate, however, that Relax runs perfectly well on these platforms.

It's only as accurate as your log files. In practice this means that the answer in no, because:

• All clients (browsers for example) don't send necessary information. According to HTTP specification (RFC 2068, section 14.37) Referer (sic) field is not mandatory.
• A person typed in the URL, which means they came from nowhere.
• A person uses Internet Junkbuster proxy to send a fake referer with every request.
• A person has started Lynx with -noreferer or -nofilereferer to disable transmissions of Referer headers.

• Partial GET requests (206), Permanent and Temporary redirects (301 and 302) can generate multiple, near identical log entries. Relax is not smart enough to spot these cases and the results can be inflated.

  Partial content requests are usually generated by download managers to speed the downloading of big files and Adobe Acrobat Reader to fetch document page by page. Redirects can be caused, for example, if the "/" character is missing from the URL.

Sometimes browsers simply mess things up and send completely wrong information. So, don't be surprised if you find bogus links from sites like http://www.washingtonpost.com/, http://www.cnn.com/ or http://www.playboy.com/ directing to your site.

A typical 256 MB desktop system can easily handle log files that can be hundreds of megabytes in size. Relax is not the fastest log analyzer, but the more memory on the computer, the better performance can be expected.

Search engine business is constantly in the state of flux. This means that Relax must be regularly upgraded to recognize new search engines and modifications made into existing ones.

Starting with Relax 2.5 the process to upgrade Relax became a lot easier than it used to be. Now you can download more recent parser module from http://ktmatu.com/software/relax/ and copy it into the directory where you have installed relax.pl or relax.exe. Updates are available quite often, usually once a week.

Relax for Windows supports new Relax Update Service. This service makes it easy to get parser updates.

Your Engines.pm file is rather old and must be updated to get accurate results. See "How to keep Relax up-to-date?" for more information.

Taken from alt.folklore.computers:

Real Programmers don't do documentation. Documentation is for simps who can't figure out the listing.

Real Programmers don't write specs -- users should consider themselves lucky to get any programs at all, and take what they get.

Tom Van Vleck <vanvleck_tom@tandem.com>

(10/25/82)

Yes. They are briefly explained in the relax.cfg file.

To take the engine report as an example, positive values like $command_line {"engine_report"} = 3; (or -re3) indicate that only those search engines which generated three or more visits are listed, e.g.

        1395    Yahoo!
        504     AltaVista
        82      GoTo.com
        54      HotBot
        ...
        5       Direct Hit
        5       Search.com
        4       canada.com
        3       Inference Find
        3       MegaSpider
        3       SavvySearch
        ...14 (17)
        =======
        2438
      

Negative values like $command_line {"engine_report"} = -3; (or -re-3) instruct Relax to print only the three most frequently used search engines, e.g.

        1395    Yahoo!
        504     AltaVista
        82      GoTo.com
        ...37 (457)
        =======
        2438
      

Please remember to check that

$command_line {"all_reports"} = 0;

before you make any other modifications!

• RefererLog:
  

  http://www.example.com/foo/bar.html -> /~mysite/index.html

• NCSA extended/combined TransferLog:
  

  some.example.com - - [05/Oct/1998:09:08:46 0200] "GET /mysite/index.html HTTP/1.0" 200 1854 "http://www.referer.int/foo/bar.html" "Mozilla/6.01Gold (Commodore 64; I; 8bit)"

• Some variations of Combined log:
  

  some.example.com www.ktmatu.com - [02/Jul/2001:08:18:21 +0300] "GET /software/relax/ HTTP/1.1" 200 37870 "http://www.referer.int/foo/bar.html" "Mozilla/4.0 (Sinclair ZX-80; I; 8bit)" -

• WebSTAR:
  

  03/03/1999 17:03:50	123.123.123.123	www.mysite.com			GET	/	http://www.referer.int/foo/bar.html		Mozilla/6.01 (Commodore 64; I; 8bit)	200	13876	60          

WebSTAR code is still quite untested.

• It's possible to define an ad hoc custom parser in the configuration file (see custom_parser). For example, if you are running Microsoft IIS web server (why?) you can try something like:

  $command_line {"custom_parser"} = \&parse_iis;
  
  sub parse_iis {
      my ($log_line_str) = @_;
  
      my ($referer_url) = "-";
      my ($target_url)  = "-";
      my (@parts)       = ();
  
      if ($log_line_str !~ /^#/) {
          @parts = split (/\s/, $log_line_str);
  
          $referer_url = $parts [10];
          $target_url  = $parts [7];
  
      };
  
      return ($referer_url, $target_url);
  };
  

  Remember to change the $parts [..] values to correspond the positions of cs(Referer) and cs-uri-stem in your log format. For example, if the #Fields: identifier at the top of a log file looks like:

   #Fields: date time cs-uri-stem cs(Referer)
  

  These variables should be defined as $target_url = $parts [2]; and $referer_url = $parts [3];. Here date is the 0th field, time = 1st, cs-uri-stem = 2nd and cs(Referer) = 3rd.

• Try to find a log file translator. LogTran is one, but it only works on Mac and is not free. Please let me know if you are aware of other similar programs.
• Send me a sample of your log. With luck I can figure out the format and make the necessary modifications.

First you have to pick your own entries from the log file. This can be done with Unix rooted utilities like grep, fgrep and egrep.

Let's suppose that the top-level URL of your site is http://www.example.com/~somebody/ and the log file is located at /www/logs/referer. Now you can use something like this to extract your own information:

% grep somebody /www/logs/referer > mylog
% relax.pl -ra1 mylog

or

% grep somebody /www/logs/referer | relax.pl -ra1

Please be patient! Results will not appear immediately as those log files can be huge in size.

So, you have a monthly/weekly log file, but you are only interested in what happened to your site during the weekend. Try something like:

% egrep "(07|08)/Oct/" log-file | relax.pl - -crelax.cfg -oreport.html

Archive file feature significantly speeds up the generation of historical reports incrementally over time from a set of log files. This feature also reduces the storage requirements, because old log files are no longer needed. Instead of crunching old log files many times, Relax can write results into a machine readable archive for later use.

To generate a historical report historical-report.html containing data from log-file-2002-01 and archive.dat it's possible to try something like:

% ./relax.pl -crelax.cfg -ohistorical-report.html -aarchive.dat log-file-2002-01

A month later it's possible to create a new historical report, now containing data from log-file-2002-01, log-file-2002-02 and archive.dat:

% ./relax.pl -crelax.cfg -ohistorical-report.html -aarchive.dat log-file-2002-02

It's also possible to generate other reports directly from an archive file, without specifying any log files:

% ./relax.pl -ra1 -oother-report-2002.html -Aarchive.dat

NB. This feature was fixed in Relax 2.50.

Unix shell cannot recognize "~" (home directory) because it is not the first character. Instead of this:

% relax.pl -c~/Relax/relax.cfg ...

use absolute filenames or $HOME like here:

% relax.pl -c$HOME/Relax/relax.cfg ...

It indicates that the referring URL was unknown, the page was loaded from user's bookmark file, the address was manually typed, or the page was fetched by a robot or crawler. Nowadays many browsers can also be configured to not send any referrer.

Try something like this in your configuration file:

$command_line {"exclude_refs"} = '^\-$';

It is one of those "great" Microsoft inventions. If someone using Internet Explorer or other browsers supporting this feature bookmarks one of your pages, the browser also tries to get an icon, favicon.ico, from your site. If found, this icon is shown next to page's title on the bookmark list.

Previously it was possible to use favicon.ico as an indicator that someone had actually bookmarked the page. Nowadays browsers like Phoenix and other applications like UCmore fetch this file without any connection to bookmarks.

For more information about favicon.ico, see:

• http://msdn.microsoft.com/workshop/Author/dhtml/howto/ShortcutIcon.asp
• http://www.favicon.com/

No, but if you are somewhat paranoid, it is also possible to run Relax off-line on a different computer.

Click a keyword in Relax reports. As a result Etsin page opens with all necessary data transferred from a report (if this fails or the data is incorrect, please check the installation guide for more information). With one click you are now able to check what kind of search results these keywords could have given in different search engines, how popular your page is, and how to improve it.

If your page is not well positioned, you can use those dictionary links and search term suggestion services to find out related words and stress them on your page.

You can also get design suggestions if you click links in the page checker section.

Referrer Log Analysis X. I haven't yet found any rational explanation for the letter X.

Nothing, Relax is absolutely free in all senses - it is freely redistributable, and no payment is required. However, I strongly encourage you to let me know that you are using Relax. The easiest way to do this is to join the Relax announcement mailing list (to subscribe, send a blank email message to <relax_announce-subscribe@yahoogroups.com>). This moderated, very low-traffic mailing list is just for major announcements.

I am also available for private consultation. If you need assistance with technical tasks such as Relax installation and configuration, managing and analyzing traffic data or other search engine related issues, please don't hesitate to contact the author, Matti Tukiainen.

If you like Relax, you can also visit The Hunger Site to give food for free to hungry people in the world.

You can also support Relax development by donating money.

All these things are totally optional - there is no pressure or obligation for you to do anything.