Python & CGI Programming


Referrer System Technical Explanation

The components of the Referrer System are listed, followed by an explanation of what the components do and how they work together.

Referrer System Components

The Referrer System components include three MySQL database tables, two JavaScript modules and four Python modules.

You can download them by right-clicking on the component names, and choosing the "Save Target As..." or "Save Link As..." option. Be sure to change the .txt extension to either .cgi or .py before you save the Python modules. Depending on what browser you use, you may need to save the Python modules as .txt, and rename them later.

My website host requires that Python executable modules end in .cgi, rather than .py. I had to change the extension of the .cgi and .py modules to .txt so you could download them from the web site. Be sure to change the .txt extension to either .cgi or .py before you save the Python modules. Depending on what browser you use, you may need to save the Python modules as .txt, and rename them later.

The SQL provided in referrerssql.txt will create a MySQL "referrers" database, and three tables: domain, referrer, and referrer_count. You can generate a development database using this SQL. The database name would have to be modified to whatever database name your website host provider gives you for your deployment database.

The JavaScript modules should be stored in the same directory as your website html. The Python modules would be stored in your CGI or cgi-bin directory. My website host defined the CGI directory as a child directory of the main website directory. If your website host has a different configuration, you may need to make some changes to the Python modules to reflect your particular CGI configuration.

Referrer System MySQL Database

I defined three MySQL tables to be used in the Referrer System. The reason that there are three tables is that I normalized the data to eliminate data duplication, and to reduce the total database size to the minimum possible.

The first table, the domain table, stores the domain of your web site, and gives it an ID number. Since these tables are used for multiple domains on the Python & CGI Programming web site, it reduces the size of the database to assign them an ID number.

The second table, the referrer table, stores the referrer URL and title, and gives it a ID number. Since there will be many referrer web sites, it reduces the size of the database to assign them an ID number. The title of the referrer is retrieved from the referrer web site. If the referrer web site has no title, the domain name from the URL is used.

The third table, the referrer_count table, keeps track of the referrer count using the domain ID and the referrer ID. A timestamp also is saved so that the Referrer System knows when the latest referral took place.

I used the InnoDB database engine because of its transaction capabilities. You can change this engine to whichever MySQL database engine you prefer.

Gathering Referrer Information

Referrer Information is extracted and stored in the MySQL database through the following process:

Routines to save referrer information

referrers.js is called by the following JavaScript code that the web site owner puts at the bottom of the .html page.


<script type="text/javascript" language="JavaScript"
src="http://www.python-cgi-programming.com/referrers.js">
</script>

referrers.js gets the JavaScript variables document.location & document.referrer from the .html page, formats them, and passes the formatted information to the ReferrerPut.cgi Python program.

ReferrerPut.cgi gets the formatted location and referrer information, and passes it to various routines in ReferrerFunctions.py. The routines in ReferrerFuctions.py perform the following functions:

Displaying Referrer Information

Referrer Information is displayed from the MySQL database through the following process:

Routines to get referrer information

referrerlist.js is called by the following JavaScript code that the web site owner puts on the .html page where he wants the referrer list to be displayed.


<div style="overflow:hidden;text-overflow:ellipsis">
<script type="text/javascript" language="JavaScript" 
src="http://www.python-cgi-programming.com/referrerlist.js">
</script>
<script type="text/javascript" language="JavaScript">
<!-- Set referrer list variables and call list function
var min = "1"	/* Minimum number of references to be listed */
var exp = "90"	/* Number of days since last reference */
var lim = "0"	/* Maximum number of references to list 
    ( 0 = list all references) */
write_ref(min, exp, lim);
// -->
</script>
</div>

referrerlist.js gets the JavaScript variables min, exp, and lim from the .html page, formats them, and passes the information to the ReferrerList.cgi Python program. The variables are set on the web page, rather than in referrerlist.js, so that the web site developer can create more than one referrer list with different variable values, if he wishes.

ReferrerList.cgi gets the formatted information, and passes it to various routines in ReferrerFunctions.py. The routines in ReferrerFuctions.py perform the following functions:

The rows are retrieved from the MySQL referrer database in Referrer_Count descending, Referrer_Title ascending order.

The JavaScript passed back to the .html page is a document.write string that generates HTML looking something like this:


<div class="rhead">Referring Web Pages, last 90 days:</div>
<div class="rname"><a href="http://www.computer-chair-traveler.org/">
Computer Chair Traveler - Scenic Pictures from Around the World</a> [13]</div>
<div class="rname"><a href="http://search.msn.com/">
MSN Search</a> [3]</div>
<div class="rname"><a href="http://www.alltheweb.com/">
AlltheWeb.com</a> [1]</div>
<div class="rname"><a href="http://search.ninemsn.com.au/results.aspx">
ninemsn Search:</a> [1]</div>
<div class="rname"><a href="http://search.yahoo.com/">
Yahoo! Search - Web Search</a> [1]</div>
<div class="rcredits">
<a href="http://www.python-cgi-programming.com/referrers.html">
List referrers to your own site (free!)</a></div>

Since the information retrieved from the MySQL database is dynamic, the referrer listing may change every time you refresh the web page.

Referrer System Changes, Suggestions

If you have any suggestions for improving the Referrer System, you can write me here.

Previous Page: Referrer System, Top Page: Index