How to archive and purge FocalScope data
Monday, December 5, 2016 2:57 AM

To help keep your FocalScope instance running at peak performance, you may occasionally want to archive and purge older or rapidly growing data from your FocalScope database. FocalScope includes functions for just that reason. Archives can be created as large files or split over a series of smaller files. Additionally, the archive routine can be set to run per schedule. 

Archives can also automatically be purged from the FocalScope database if the purge option has been configured in the archive routine, or when using the global purge configuration parameters. This article details how to configure and use FocalScope's archive and purge functions.

Disclaimer: Once the archived data has been downloaded from the FocalScope server, it becomes the sole responsibility of the customer to maintain and secure the data. FocalScope cannot be held liable for what happens to data that is not retained in FocalScope's database.

For a diagrammatic overview of the archive and purge functions, please download the attached PDF file.

Tutorial

Understanding the archiving process

During the archiving process, the following FocalScope data is archived:

  • Incoming emails
  • Replies to emails (outgoing)
  • Post-to-folder messages
  • SMS messages
  • Autoresponder emails

Warning, the moment a data archiving routine has been configured and started, it is no longer possible to edit, stop, or delete the archiving routine. This is necessary because changing archiving parameters could affect data purging (if enabled) and to ensure that an accurate auditing trail is left for all data archiving operations.

Configuring the archive function

  1. In the [Main menu], select [Screen > Administration]
  2. Click the [Advanced Properties] tab (highlighted in red)
  3. Navigate to the [Maintenance > Data archiving] folder
  4. At the bottom of the right-hand pane, click [New] (see Figure 1)

The following explains each setting in the [Archive - Properties] page:

  1. General tab - select this tab to configure the archive routine
  2. Files tab - select this tab to view and download completed archive files
  3. Name - specify a name for your archive routine here
  4. Recommended size - set the approximate archive file size. When the archive file is being created and it reaches the specified size, a new archive file will be created (archive is split over multiple files). Please note, some emails may have large attachments that cannot be split over multiple archive files, as such, the size limit may be exceeded in some archive files
  5. Purge archived data - set the purge function behavior for the archive data. Warning, there is no way to reverse a data purge! Use this option with care. The options are: always, never, only if global purge is on (see the 'Configuring the purge function' section further down in this article for more details on this option)
  6. Processing status - indicates the state of the archive routine. Please note, active routines cannot be modified. States are: active, completed, paused (if perpetually paused it's most likely that errors occurred during the archiving process)
  7. Zip archived files - adds each archived .db file to its own zip archive
  8. Filtering options - specify which personal and / or shared folders the archive function should operate on. You can also set the option to include all sub folders of selected folders
  9. Date range - set the date range you want the archive function to operate on. If you want to archive only emails that are a year old, set the date range accordingly
  10. Ticket state - set the ticket state (open, on hold, closed, or any) that you want the archive function to archive
  11. Category - specify what category and / or value of tickets the archive function should archive. Query format: category=value (value is optional)
  12. Exclude deleted - set the archive function to include or exclude deleted tickets
  13. Click [Save] once you have configured the archive routine


Figure 1 - Archive routine settings


Configuring the archive scheduler

Next you need to configure a schedule for when the archive routine should run. It is recommended that the archive schedule be set to after office hours, when the impact on the system will not affect users. 


  1. Click the [Schedule] tab (see Figure 2)  to access the archive schedule page
  2. Click [New] to configure an archive schedule. Please note, you can have multiple archive schedules operating at different times of the day (it is best if these time frames don't overlap)

The following explains each setting in the [Schedule - Properties] page:

  1. Time from & Time to - set the time frame (24-hour format) during which archive routines will run
  2. Interval (min.) - set the interval (in minutes) at which the purge routines will archive data. Please note, a lower interval has a greater performance impact on the system
  3. Items per batch - set the number of tickets / emails to be processed per interval. Please note, to prevent performance degradation on systems that operate 24/7, this setting has an upper limit of 200 items per batch
  4. Click [Ok] to save the configured archive schedule


Figure 2 - Archive schedule settings


Configuring the purge function and purge schedule

The purge function lends itself to the archive function in that it can automatically delete archived data from FocalScope's database. Automatic purging of redundant / outdated data is highly recommended to keep FocalScope performing at optimal levels.


  1. In the [Main menu], select [Screen > Administration]
  2. Click the [Advanced Properties] tab (highlighted in red)
  3. Navigate to the [Maintenance > Data purging] folder

The following explains the settings of the purge function as they relate to the archive function:

  1. Automatically purge archived emails - tick this flag to use the purge function in conjunction with the archive function (this option correlates with the [Purge archived data] function under the 'Configuring the archive function' section at the start of this article). 
    1. Click [Save] to apply changes
  2. Under the [Schedule] section, click [New] to create a new purge schedule

The following explains each setting in the [Schedule - Properties] page:

  1. Time from & Time to - set the time frame (24-hour format) during which purge routines will run
  2. Interval (min.) - set the interval (in minutes) at which the purge routines will archive data. Please note, a lower interval has a greater performance impact on the system
  3. Items per batch - set the number of tickets / emails to be processed per interval. Please note, to prevent performance degradation on systems that operate 24/7, this setting has an upper limit of 200 items per batch
  4. Click [Ok] to save the configured purge schedule

Figure 3 - Purge function settings

Monitoring archiving and purging progress

By looking at the statuses highlighted in blue for the columns highlighted in red, you can determine the progress and estimated completion time for active archiving and purging routines. Please note, any estimate has to be subject to potential variability, because completion time of archiving and purging routines depends greatly on how much system resources are occupied at during the time archiving and purging processes are running.



Figure 4 - Monitoring archiving and purging progress


Downloading archive data files

Once an archive routine has run, it will show a completed state and indicate how many items were archived (highlighted in Figure 5).

Disclaimer: Once the archived data has been downloaded from the FocalScope server, it becomes the sole responsibility of the customer to maintain and secure the data. FocalScope cannot be held liable for what happens to data that is not retained in FocalScope's database.

To download the archive files, do the following:


  1. Select the archive routine whose archive files you want to download and click the [Properties] button
  2. In the [Archive - Properties] page, click the [Files] tab. Displayed in the list are the archive files with their relevant details
  3. Select the archive file you wish to download and click the [Download] button


Figure 5 - Downloading archive files


Additional Information

How to use the FocalScope ArchiveViewer



Attachments
Data Archiving And Purging Diagram.pdf