Documents
Features
READ ME
Sample configuration
ChangeLog
Online Docs

Source code
Download
Browse
Bazaar

Mailing list
Bug Tracker

Smart Cache Loader on
Freshmeat page
SForge page
Freshports
Ohloh

Stats
SForge
Freshmeat
CNW

My projects
Smart Cache
SC Loader
Download Machine
FSP Suite
FSP Client
Other programs

Smart Cache Loader Features

Design goals

1. Non graphical
2. Non interactive
3. Overkill configurable
4. Portability (uses Java)

Data storage for downloaded pages

1. null - do not store, used for proxy preloading.
2. local - write pages to filesystem
3. scache - used when working with Smart Cache

Priority based queue

1. URL in queue can have different priority by type, size, ...
2. Servers can have different priority

Fetch actions

1. do not download
2. download but do not parse html code
3. send request, but do not wait for any data
4. classic full download
5. fastclose - if data are not parseable, close connection
6. nosave - do not save it

Update actions

1. if object exists, do not re-download it
2. for cache reload of specified object
3. update object if older than XX hours
4. do not reparse saved html on disk
5. forced update

Target categories

1. world - any server
2. known - any server in configuration file
3. file from the same server
4. file from the same location (directory+subdirs)
5. file from subdirectory
6. update object if older than XX hours
7. forced update

Locations

1. You can define multiple location of the same data for avoiding duplicate download
2. Can define short name for server for use from command line.

Masking

1. URL mask with possible striping before comparing
2. File extension mask
3. Source HTML tag mask
4. Target selection mask