This tutorial will show you how you can implement a system into your web site that will notify you when a search engine robot is crawling your site. The first thing you do is to create a variable called "stringToSearch" that contains the visitor (or robot) browser system (or better known as USER_AGENT).
<cfset stringTosearch = cgi.USER_AGENT>
Next, we will look to see if it it a real browser or not:
<cfif (findnocase("MSIE",stringToSearch) EQ 0) AND
(findnocase("Gecko",stringToSearch) EQ 0) AND
(findnocase("Opera",stringToSearch) EQ 0) AND
(findnocase("Konqueror",stringToSearch) EQ 0) AND
(findnocase("Safari",stringToSearch) EQ 0) AND
(findnocase("Netscape",stringToSearch) EQ 0)>
<!--- THE VISITOR IS NOT USING A BROWSER, SEND AN EMAIL ALERTING ME ITS A ROBOT CRAWLING MY SITE --->
<cfmail to="bots@yourdomain.com"
from="bot#cgi.REMOTE_ADDR#@yourdomain.com"
subject="Spider Bot Alert"
type="HTML"
server="mail.yourdomain.com">
<font face="verdana" size="2">
<p><a href="http://ws.arin.net/cgi-bin/whois.pl?queryinput=#cgi.REMOTE_ADDR#">#cgi.REMOTE_ADDR#</a></p>
<p>#cgi.HTTP_USER_AGENT#</p>
<p><a href="#cgi.HTTP_REFERER#">#cgi.HTTP_REFERER#</a></p>
</font>
</cfmail>
</cfif>
-
Aliasing Your SQL Statements
Many developers I have talked to are not aware of the ability to create an "alias" in an SQL statement, concatenate, and even add strings into your SQL statements. A little work up front in the SQL can cut out a lot of tedious coding in thetag.
Author: Jim Summer
Views: 14,975
Posted Date: Tuesday, December 10, 2002 -
Database Dates (between ranges)
This deals with database dates: (1)inserting a properly formatted date into the database, and then (2)pulling a query from the database between a date range defined by 2 text boxes.
Author: Jim Summer
Views: 17,945
Posted Date: Monday, December 9, 2002 -
Navigation as an include file
Create an include file (custom tag) for your navigation to help make maintenance easier! This include file lights up the button depending on where the user is at in your website, and is XHTML 1.1 validated! If you need to edit your navigation, just do it in 1 place, the include file! Javascript included :)
Author: Jim Summer
Views: 52,843
Posted Date: Thursday, December 12, 2002 -
Recordset Paging in Cold Fusion
This will pull a predefined number of records from a database, allow the user to change the number of records to be shown, and write the "NEXT" or "BACK" (or both) buttons at the bottom of the page. Thus allowing the user to "surf" through the database. See it in action at http://freecfm.com/t/tentonhead/ and click on the "CTCS" link when you get there. The HTML for this page has changed a little since I first did this (so it validates as XHTML1.1) but the CFML remains as it is in this snippet.
Author: Jim Summer
Views: 26,403
Posted Date: Monday, December 9, 2002 -
Replacing "enter" key with "<br>" tag
This little piece of code will transform those pesky "enter" keys in a textarea into "
" tags so your users input is printed out properly in your html page.
Author: Jim Summer
Views: 17,740
Posted Date: Friday, December 13, 2002 -
Search Engine Bot Notifier
This code detects the most common user agents (web browsers) and notifies you via email if it is not a recognized user agent as defined in the code. Usually this will be a bot of some sort. Extrememly useful for tracking how often Google, Yahoo, etc visits your site. It will email you the bot and a reverse IP lookup url with the IP appended so you can verify if it is a "good bot" or a "bad (spam) bot" (then you can block that IP or stop processing of the page). Use this in your home page or as an include file throughout your site. Nothing fancy it should work on MX also although I have not tested it there.
Author: Jim Summer
Views: 11,526
Posted Date: Wednesday, April 14, 2004
