Folder Scanning Event – Cerberus Support

The Scan A Folder workflow is broken into two steps. A Scheduled Task and an Event Rule. The Scheduled Task action allows a user to manipulate every file in a folder on a regular schedule without needing to know the specific files in the folder.

Add a folder path to a Scan A Folder scheduled task action and, for each file in that folder, the task will trigger File Scanned Events you have defined in Event Rules. These events can trigger Event Rules to perform operations on the files in the folder. With this event’s ability to specify a condition on the generating task’s name, the rule can be restricted to only occur for a specific Scheduled Task.

Example Scan A Folder Overview

These instructions show how to set up the Scheduled Task and create and add the appropriate conditions for a Scan A Folder workflow. The example actions are for a workflow if you want to scan a remote folder on an SFTP server and get the files in the remote folder. This might be a case of automating daily downloads at midnight and archiving the files locally, then deleting the files off the remote location. You can also scan a local folder, or a remote folder on an FTP or FTPS server. Other action examples can be found in Rule Examples and Creating and Editing Event Rules. as well as the Event Manager Rule Examples section of our Online Help.

This example has a scheduled task that triggers every night at midnight. The Scan A Folder action looks at the uploads folder and, for each file there, sends out a File Scanned Event. These events are processed in order by the Events System, each time looking for rules that trigger off of the given File Scanned Events. In this example, the first action retrieves the scanned files. The second action moves the downloaded files to a local archive folder. The third action deletes the files on the remote location so that they are not found again the next time the scheduled task runs. If one of the actions (file get/moves/delete) fails due to an error, the Failure Action will send an email to notify the administrator.

STEP 1: Define the Scheduled Task

The first step to using this feature is to define when this action is to occur. Navigate to Event Manager > Scheduled Tasks and click the 'New' button to add a new task. Note down the name you give the scheduled task as you will need it later on when you create the event rule.

Define a time for this task to occur, and how often it will repeat. In this example case, the action will occur daily at midnight.

Create a new action by using the new button and choosing Scan A Folder in the Action: drop-down menu.

This action has the same top elements as all other actions: An action type and ‘using’ field gets filled in for Scan A Folder and is not editable and the standard “Stop on Failure” checkbox as all actions have, which determines if processing will continue to the next action if this action fails.

The first subsection is “Fail If.” This section helps define what Scan A Folder should consider a terminal failure state, and when it will ignore and continue to process the next action. Is a Failure state one where it can’t create an event for every file in the given folder? Or is it a failure state only if every file scanned fails, continuing if there is even only one found file? Or, if nothing is processed, then perhaps that is a failure state where you don’t want to send a confirmation email if nothing was scanned. There is even the case where this is a failure if any one of the event rules that handled the new File Scanned Event returned an error. These options all work together to determine if this action failed in terms of the prior checkbox.

“Scan Folder Path” is the directory you’d like to scan. This path must be a directory or, in the case of a remote location, you can leave it blank for the root location. Cerberus FTP Server will create a File Scanned Event for each file in that directory. If you’d like not only the defined folder, but all of its children, simply check the “Recursively transfer all sub-folder content” box.

The filter string is a way to pick up only a single type of file. It will append the filter string to the folder path if it’s in the form of * for a wildcard, and ? for single characters. For example, *.txt will create events for every text file in the directory.

In the example provided, the remote server's root location is the target folder, with either an empty filter string or a wildcard * in the filter string. This creates a list of File Scanned Events, one for each of the files in the remote folder.

STEP 2: Define the Event Rule

Next, you will create a rule that will be triggered off of the Scheduled Task. Navigate to Event Manager > Event Rules and click 'New' to add a new rule. Then, select a File Scanned Event rule type.

Add Rule Conditions

To tell Cerberus which rule to trigger when the Scheduled Task runs, in the Rule Conditions, use the {{TASK}} variable == (equals) the name of the “FolderScan” Scheduled Task you set up above.

This task name check may also check to match more than one task name. For example, setting the Matches These Conditions to Match if ANY Filters Match and setting a “RunMeDaily” Scheduled Task, and a “RunMeWeekly” Scheduled Task name. In this example this rule is triggered by two different Scheduled Tasks, one that runs every day and one that runs once a week.

Available Variables

The above example is a very simple rule that pulls all files from a root folder to a location local to your Cerberus server. Filtering on which files are matched by the Scheduled Task can be limited by taking advantage of the suite of variables available for use with a Scan A Folder scheduled task and associated rule.

For example, in the case of recursive folders and/or files, the {{SETCNT}}, or ‘Set Count’ variable, contains the number of files in the Scan A Folder action and the {{SETLAST}}, or ‘Set Last File’ can be used to trigger the action on the last file found.

The {{TARGET}} variable contains the label from the Transfer Event Target used in the Scheduled Task. This matches the label you gave your Event Target when you set it up in 'Event Targets. This variable can be useful information to include in any emails sent about scanned files when creating an event rule referencing files scanned from a remote system.

Complete list of the variables for this rule type (as of Cerberus 2024.2.0):

Variable	Type	Description
{{T}}	Date and Time	Timestamp in Unix Time
{{LFN}}	String	File Name
{{LPNT}}	String	Local File Parent Directory
{{LFP}}	String	Local File Path
{{RFP}}	String	Relative File Path
{{FRFP}}	String	Full Remote File Path
{{TARGET}}	String	Operational File Transfer Target Label
{{DIR}}	Boolean	Is a Directory
{{SZ}}	Integer	File Size
{{TASK}}	String	Task Name
{{SETCNT}}	Integer	Size of Scanned File Set
{{SETPOS}}	Integer	Position of Scanned File Set
{{SETLAST}}	Boolean	Last File of Scanned File Set
{{TCRT}}	Date and Time	File Creation Time in Unix Time
{{TCRTSTR}}	Date Filter String	File Creation Filter Value
{{TACC}}	Date and Time	File Access Time in Unix Time
{{TACCSTR}}	Date Filter String	File Access Filter Value
{{TMOD}}	Date and Time	File Modification Time in Unix Time
{{TMODSTR}}	Date Filter String	File Modification Filter Value
{{LAST1HR}}	Date Filter String	File Value for 1 Hour
{{LAST24HR}}	Date Filter String	File Value for 24 Hours
{{LAST7D}}	Date Filter String	File Value for 7 Days
{{LAST30D}}	Date Filter String	File Value for 30 Days
{{LAST365D}}	Date Filter String	File Value for 365 Days

Using Time Based Conditions

There are 2 broad categories into which the time based variables can be divided:

Absolute Date: Absolute time variables are used when scanning for files older than a specific date by making use of Date Picker on the user interface. See more details of their exact usage below.
List of absolute time variables:
- TCRT (Time Created)
- TACC (Time Accessed)
- TMOD (Time Modified)

Examples of usage:

I) Scan for files whose creation time is before 15th April.

II) Scan for files whose modification time is after a specific date-time.

Filter Focused: As the name indicates, these are time variables used when scanning older files relative to current time. For example, they could be used to scan for files which are 3 weeks older with respect to current time. These provide greater control to the users by making use of time expressions. More details on their exact usage is below.
List of absolute time variables:
- {{TCRTSTR}} - Time Created Structure
- {{TACCSTR}} - Time Accessed Structure
- {{TMODSTR}} - Time Modified Structure

Examples of usage:

I) Scan for all files which were created before the last hour.

II) Scan for all files modified 1 year past their creation time.

There are relative time variables which represent the most commonly used time periods. These variables are used to scan older files using relative time by using simple time expressions. The variables are listed below:

{{LAST1HR}} - One hour in seconds
{{LAST24HR}} - 24 hours in seconds
{{LAST7D}} - 7 days in seconds
{{LAST30D}} - 30 days in seconds
{{LAST365D}} - 365 days in seconds

Example of usage:

I) Scan files older than 30 days(using creation date):

To take advantage of these variables, provide any simple math expression as the filter value when using relative time variables. The value provided must only contain numbers or numeric based event variables (eg. {{T}}, {{SETCNT}}, {{TMOD}}). Numeric based event variables should be entered manually in the ‘Value’ field.

Note:

i) Only addition and subtraction operations are allowed on the numerical values.

ii) Any other non-numerical values or other operations like multiplication, division etc. are not currently supported.

Examples of filter expressions:

Filter Expression Validity

{{T}}-abcd - Invalid Expression (non-numeric)

{{TASK}}-{{T}} - Invalid Expression (arithmetic operation on strings)

{{T}}%20 - Invalid Expression (HTTP encoded character)

{{T}}-{{LAST30D}}-3600 - Valid Expression (valid arithmetic formula)

The following use-cases are provided to allow for better understanding of the feature usage and the purpose of absolute & relative time variables.

I) Use case #1: You want the rule to perform some operations only on files which existed before a given date. In this case you can make use of the absolute time variables to look for all files which were present before the defined date. Add a condition using the TCRT variable and a date. Either use the date picker or type in the date:

II) Use case #2: You want to run a scheduled task which will scan all files which have not been modified for more than 1 year (365 days). In this case, make use of relative time variables in a simple math expression to find such older files. Either of the two condition types can be specified to scan for such files:

Condition using the predefined 'LAST365D' variable:

Alternative Condition using number of seconds in 365 days:

Note: LAST365D represents 31536000 seconds (number of seconds in 365 days) so they can be used interchangeably.

Once you have defined the rule conditions, in the 'Action' section, you can define what you would like this rule to do. See Creating and Editing Event Rules for more details.

Add Rule Actions (SFTP Get Files Example)

Add the Event Target

The Transfer File Target in Event Manager allows transferring files to and from other servers via SFTP, FTP, FTPS, or HTTP/S PUT. With this feature, there is integrated functionality making it easy to send or get files and capture any messages directly within Cerberus.

To use the functionality, start by creating a new Event Target. Refer to this document for instructions on how to create the Event Target: Event Manager: Add a File Transfer Event Target for FTP, FTPS ,SFTP or HTTP/S PUT/GET

Add the Rule Actions

Add a new action and select Get a File in the 'Action' drop down. In the Using drop-down, select the Event Target created in the Event Targets step. See the ‘Get a File Event Action’ screenshot below.

Since the action is getting a file that just came in, we can set File Path From to {{FRFP}} which is a variable representing the 'Full Remote File Path.' And for File Path To, we can enter the folder path and the {{LFN}} variable, which is just the filename part of the incoming file.

Clicking Add will complete the Rule and show a summary like:
SFTP File Transfer Target: Get file from "sftp://ibutteriss@192.168.4.164/{{FRFP}}" to "C:\ftproot\Company\{{LFN}}"

Send a File Event Action

Since the Scheduled Task will continue to find the same files over and over in the remote location unless they are removed, it is recommended to have the admins of the remote location remove the files to be downloaded once your Cerberus logs in and grabs them. There currently is no way to use Cerberus to delete files from a remote location. It is possible to create a script to do so. Reach out to Cerberus Support for more information.

If you have files available for download in the remote location, you can test the download. Turn on both the event rule and the Scheduled Task, then press 'Run' on the Scheduled Task to do a one time test.

If you have any issues, make sure DEBUG is enabled on the 'Log' page of your Cerberus desktop console ('Log' > press the bug icon on the top toolbar), trigger the transfer, and look at the debugging and error messages. Reach out to Cerberus Support and send the Logging in your ticket request. To copy logging, press the clipboard icon next to the bug icon on the top toolbar and past into a Notepad document. Include the Notepad document in your support request to reduce time to resolution.