Extranets - Database-driven sites

A database-driven web site provides the widest range of extranet-related possibilities. This page discusses:

Whether you need a database-driven web site, and how you link to it as part of your extranet will largely be determined by:

  • whether your web server is in-house or hosted by an ISP
  • the objectives of your extranet
  • the level of interactivity you think you will require
  • the degree of automation required to minimise ongoing costs
  • the time delay you are prepared to accept between something happening on your web site and your internal systems knowing about it
  • your in-house information processing workflows

Extranet development technologies

Data-driven web sites typically use a combination of HTML pages (with or without JavaScript), e-mail, forms and PERL and/or PHP scripting to create a "front end" for the database and manipulate the data.

Products such as Cold Fusion - our preferred database web site development tool - allow us to create systems that "react" to the values stored in the site database as a result of visitor activity. In turn, these reactions interact with products like Cold Fusion to provide a more personalised experience for the site visitor - i.e. the visitor drives the database and the database determines the the options that are presented to the visitor.

For businesses used to running Microsoft products we can provide data-base-driven sites using Microsoft Access and/or Microsoft SQL Server. For clients that prefer Unix/Linux we tend to use MySQL. There are many many alternatives to these database tools - at the end of the day the choice of database depends on whether you want to run with MS-Windows technology or Unix/Linux technology.

Data Uploading / Downloading options

Although you have a wide range of "front end" tools and "back-end" database technologies to choose from when developing a database-driven extranet:

  • Without an in-house web server (it might be physically located on the other side of the planet), you still need to consider how your internal systems will interact with the server on a timely basis in order to provide a high level of service to your clients
  • You also need to consider ways of minimising the effort required to maintain a good level of in-house task automation.

Getting data from the server to your desktop machine ready to go into your office machine is pretty much the same as it is for "Brochureware" sites. Your basic tools are:

  • Action-specific e-mail messages
  • Per Session or Persistent FTP for small files
  • PERL / PHP Scripts

Because the size of your database is likely to grow quite large, continuing to use FTP to download the entire database on a regular basis is not a feasible thing to do if your server is not on your internal network.

The most practical solution is to use automated PERL / PHP scripts to extract transactions as they occur and pull these transactions into your in-house system.

For example, you could run a download scripts every 5 minutes if you are working to tight deadlines - which means that your site data is never more than 5 minutes out of synch with your in-house database.

You end up with two versions of your database - one on your site which your ISP should back up every 24 hours - and one on your internal system which should also be (automatically) backed up at least once every 24 hours.

Important: Despite the convenience, you cannot rely on e-mail systems for high transaction volume levels. At best, e-mail can be used as a backup system for PERL / PHP scripts that periodically extract transactions from your web site database, and/or as a way of letting you know about occasional or non-critical, low transaction rate, events.

The Timeliness of Web Site Data Updates

If you need to react quickly to visitor actions on your web site then you need to give serious consideration to running an in-house web server.

In-house web servers operate in pretty much the same way as shared or dedicated servers - with one major difference - they are able to be quickly accessed as if they were part of your normal in-house network.

This means that the data on your web server can be maintained via the same database as the rest of your in-house systems. Your web site becomes an integral part of your in-house system and data updates can occur in "real" time.

This can be particularly important when web site visitors are interacting with information from:

  • Inventory / Stock Levels
  • Dynamic Price Lists
  • Live Production Schedules

Aside from the cost of hardware and systems software for your web server, the major cost is making sure that your web server is operational 24/7.

Some server technologies are more stable than others - for example there is strong evidence to suggest that Linux/Unix web servers are more stable than servers running Microsoft technology.

However, even the most stable software can stop functioning. If you are considering running an in-house server, issues like a continuous power supply and the need to set up "hot" backup servers need to be factored in to your planning. This increases the costs of doing business via the web but it can be a more cost effective solution for (usually) larger companies.

Proper system architecture and design can reduce the need to operate an in-house web server.

  • For example, if the inventory level preparing on your web site needs to be closely monitored, you could use a permanent high speed data line to a web server housed by your ISP.
  • If a leased line is too expensive you might want to consider setting minimum stock levels and automatic re-order levels for your web site inventory based on historical demand trends for each product on your web site.

To summarise, there are many ways to provide a high level of "timeliness" for your web site - the best solution only becomes apparent once you establish criteria for your site. You ignore the issue of timeliness at your own peril - if you underestimate your needs, customers will complain about your service - if you overestimate your needs, you will be spending money you don't have to.

On-site Data Maintenance

In addition you also have the option of maintaining your site directly - i.e. you can build maintenance screens and reporting options into your web site. Security options normally prevent unauthorised access to maintenance and site reporting. 

Data maintenance via on-site tools is feasible for small volumes of data, but is normally not cost-effective once your site traffic grows - especially if you have a slow Internet connection.

  • Your team has to resort to cut and paste (a.k.a. "screen scraping") techniques to move large amounts of data.
  • If you do not have a fast connection to your web site, maintenance can be a very slow process.
  • The cost of developing web-based tools may be far higher than developing a custom in-house system

One valid option is to use PERL/PHP scripts to download large volumes of non-sensitive data and "cut and paste" sensitive relatively low volume data such as credit card numbers etc.

Once traffic grows the only realistic way to maintain your web site data is to set up your own in-house extranet server and access it directly via your internal network.

See also: Server options | E-Mail Only systems | Brochureware Sites | Peer-to-Peer options

Back to Top

eCommerce - eBusiness Database Sites - Back to Remarkable Home Page
eBusiness with CustomShop eCommerce Database Sites

Unless specified otherwise, the entire contents of this site are protected by copyright.
(c) 1999-2010 Noel Ferguson / Remarkable Ideas Ltd. All Rights Reserved.
Postal Address: PO Box 633, Tokoroa 3444, Waikato, New Zealand
Mobile: +64-27-306-8999 After Hours: +64-7-886-0445