Skip to main content

Purging

Few tables in the AutomationEdge(AE) system get populated with a large amount of data. Over time, the records in these tables grow, and there is a need to purge them to run with optimal performance.

Purging allows the setup of a Purge Policy and Purge Schedule to purge these database tables as needed. It reduces database and disk space usage and, in turn, increases AutomationEdge system performance.

Purging option is available for the following data tables,

  • Audit Logs

  • User Session History

  • Workflow Requests

  • Notification History

Purging has three menu options: Purge Policy, Purge Schedule and Archives. A System Administrator has permission to set up Purge Policy and Purge Schedule. A Admin can only edit the Purge Policy at the Tenant level and view the Purge Schedule. Thirdly, the Archived menu is also visible to Admin to view and download Archives.

The sections below discuss the three menu options,

Purging Configuration​

The Purging Configuration includes the purge policy and schedules options. You can configure the details through the Policy and Schedule tabs, respectively.

Policy​

A purging policy includes a Purge Duration and Notification details. Purge Policy determines Purge Duration, in months, for each table data for Audit Logs, Workflow Requests, User Session History, and Notification History. For Workflow Execution Metrics data, the purge duration is in days. The Purged data is then available as Archives.

tip

The Workflow Execution Metrics purge details is available if the Workflow Metrics service is running.

There are two scenarios for the application of the purge policy: Scenario 1: If the flag ae.global.purge.enabled=false, in the ae.properties file, then:

  1. You can apply purging only for those tenants who have specified a purge policy.
  2. The Purge Duration set by the System Administrator and Admin are considered, and the policy which has lesser purge duration is applied to the tenant. For example, see the following screenshot:

Scenario 2: If the flag ae.global.purge.enabled=true, in the ae.properties file, then: 3. You can apply the purging policy globally to all tenants irrespective of whether the tenant has defined a purge policy or not. If the tenants do not have a purge policy, then the System Administrator’s policy is applied to all tenants. 4. If tenants have purged policy, then Purge Duration set by the System Administrator and Admin are considered, and the policy which has lesser purge duration is applied to all tenants.

Purge Duration​

Set a Purge Duration for the following for the database tables in months. The minimum duration is one month. It is mandatory to set the duration for all four tables (a value greater than zero).

Notification Details​

System Administrators or Admins may send notifications if SMTP configuration is done for them. Following are the use cases,

  • Sysadmin sends emails to all the Admins in the cases mentioned below,

  • But suppose the Tenant overwrites the purging policy. In that case, the Admin sends mail from the configured SMTP for the Tenant to all Admins in the Tenant organization and the emails listed for notifying.

The following events result in sending notifications,

  • Sysadmin creates/updates Purge Policy

  • Sysadmin creates/updates/deletes Purge Schedule

  • Tenant Admin updates Purge Policy, specific to the Tenant. AutomationEdge sends an email to Admins in the Tenant and Users emails listed for notifying.

  • Three days before the purging schedule, a notification is sent every day to all Tenants Administrators and user emails listed for notifying.

The number of days is configurable such as seven days etc. The number of days is configurable in the AutomationEdge properties file. Set ae.purge.notification.period=<number of days e.g.7>.

  • Notification after purging completion is for all Tenants.

Following are the steps to create a Purge policy, including Purge Duration and Notification Details,

  1. Navigate to the Purging🡪Purging Policy menu.

  2. An Admin can apply or edit the policy.To apply the purge policy, click Apply. Application of the tenant policy is a one-time activity.

Note: Apply is unavailable, if ae.global.purge.enabled=true, in the ae.properties file. Or To edit the tenant policy, click Edit.

tip

You can set the purge duration of the policy equal to or less than the duration set by the System Administrator. Else, you’ll see the following:

The purge policy is applied depending on the purge duration set by the System Administrator or Admin. If you set the purge duration less than that set by the System Administrator, then the Admin’s purge policy is applied to the tenant.

  1. Change Purge Duration by providing new Configuration details. Also, provide Notification Details.

  2. Click Save.The purge details are saved.

Figure 17.1: Update Policy Configuration

Schedule​

Several options and flexibility in setting up the purge schedule have been provided based on,

  • Day of a month

  • Day of a week (First, Second, Third, Fourth or Last)

Note:

  • The Purge schedule executes a minimum after seven days from the current date.

  • The first execution of the Schedule will happen from the next month if the days between now and the next trigger is less than or equal to Seven days for the current month.

  • If the server is down on the day the schedule is to be triggered, then the schedule executes the next day at the same time.

Following are the steps to view a Purge Schedule,

  1. Navigate to the Purging🡪Purge Schedule menu.

  2. Purge Schedule set by the System Administrator. Admin can only view the Purge Schedule, as seen below.

Figure 17.2: View Purge Schedule

Archived​

All the purged records are archived and exported to CSV files in batches of 10000 records.

Download the archived files from AEUI or the purging target directory. Archives menu lists Archives and is also available for download. Archives are visible and available for download only once a Purge Schedule completes execution. If a purge job creates multiple files, those many entries are visible for downloading on AEUI for the purge job. It also applies to the Purging Target to fetch Archived records.

For CONSUMPTION based tenant, the Workflow Consumption report is saved as a PDF with tenant name prefixed in the name. The report is saved in the tenant's Archives location.

The following sections discuss both the options,

Archives from AE UI​

Following are the steps to view and download Archives from AE UI,

  1. Navigate to the Purging🡪Archives menu.

  2. Archives are available once at least one Purge Schedule execution is complete. In the beginning, there are no Archives.

Figure 17.3: View Archives

  1. The Fire Time column shows the Purge Schedule execution time.

  2. Download Archives by clicking on any File Name link as seen below.

  3. Delete Archives by clicking the delete icon ( ) in the Actions column.

Archives from Purging Target​

  • In addition, AutomationEdge stores the archived CSV files as zip files in the default location, AE Home\Archives.

  • However, the default purging destination is configurable, set it in the AE_Home/conf/ae.properties by setting the following configuration parameter,

ae.archive.location = <Desired filepath(e.g D:/testPurge/Archives)>

  • Tenant folders are created in the purging destination.

The Zips files include purging data of all the three entities (Workflow Requests, Audit Logs Notification History)

Last updated on