Folder Scanning Event

This Scheduled Task action allows a user to manipulate every file in a folder without needing to know the specific files in the folder.

Administrators have previously needed to know ahead of time the names of files for use with actions such as Send a File, Get a File, file operations, and others.  Add a folder path to this Scan A Folder scheduled task action and, for each file in that folder, the task will trigger File Scanned Events you have defined in Event Rules.  These events can trigger Event Rules to perform operations on the files in the folder. With this event’s ability to specify a condition on the generating task’s name, the rule can be restricted to only occur for a specific Scheduled Task.

Example Scan A Folder workflow

These instructions show a simple example as to how a Scan A Folder workflow might occur on a local folder. This might be a case of a teacher automating their homework submission folder such that CerberusFTP takes all files uploaded to a folder by midnight and archives them. Cerberus 2024.2.0 introduced the ability to scan a remote folder on an FTP or FTPS server (SFTP to come in a future release).

This example has a scheduled task that triggers every night at midnight. The Scan A Folder action looks at the uploads folder and, for each file there, sends out a File Scanned Event. These events are processed in order by the Events System, each time looking for rules that trigger off of the given File Scanned Events.  In this example, the first rule sends scanned files somewhere, then moves the file to another folder. The second rule only triggers on the last scanned file in the folder, and, only when all files have been moved, zips up the directory for archiving. If one of the actions (file sends/moves/zipping) fails due to an error, the Failure Action will send an email to notify the administrator.  Otherwise, the location of the archive is sent by HTTP post to another server.

Defining the Scheduled Task

The Scan A Folder workflow is broken into two steps. The first step to using this new feature is to define when this action is to occur. Navigate to Event Manager > Scheduled Tasks and click the 'New' button to add a new task. 

Define a time for this task to occur, and how often it will repeat.  In this example case, the action will occur daily at midnight.

Create a new action by using the new button and choosing Scan A Folder in the Action: drop-down menu.

This action has the same top elements as all other actions: An action type and ‘using’ field gets filled in for Scan A Folder and is not editable and the standard “Stop on Failure” checkbox as all actions have, which determines if processing will continue to the next action if this action fails.

The first subsection is “Fail If.” This section helps define what Scan A Folder should consider a terminal failure state, and when it will ignore and continue to process the next action. Is a Failure state one where it can’t create an event for every file in the given folder? Or is it a failure state only if every file scanned fails, continuing if there is even only one found file?  Or, if nothing is processed, then perhaps that is a failure state where you don’t want to send a confirmation email if nothing was scanned. There is even the case where this is a failure if any one of the event rules that handled the new File Scanned Event returned an error. These options all work together to determine if this action failed in terms of the prior checkbox.

“Scan Folder Path” is the directory you’d like to scan.  This path must be a directory and Cerberus FTP Server will create a File Scanned Event for each file in that directory.  If you’d like not only the defined folder, but all of its children, simply check the “Recursively transfer all sub-folder content” box.

The filter string is a way to pick up only a single type of file.  It will append the filter string to the folder path if it’s in the form of * for a wildcard, and ? for single characters.  For example, *.txt will create events for every text file in the directory.

In the example provided, the folder “C:\ftproot\uploads” is the target folder, with either an empty filter string or a wildcard * in the filter string.  This creates a list of four File Scanned Events, one for each of the four files File A-D in the folder.

Defining the Event Rule

Next, you will create a rule that will be triggered off of the Scheduled Task.  Navigate to Event Manager > Event Rules and click 'New' to add a new rule.  Then, select a File Scanned Event rule type.

To tell Cerberus which rule to trigger when the Scheduled Task runs, in the Rule Conditions, use the {{TASK}} variable == (equals) the name of the “FolderScan” Scheduled Task you set up above.

This task name check may also check to match more than one task name.  For example, setting the Matches These Conditions to Match if ANY Filters Match and setting a “RunMeDaily” Scheduled Task, and a “RunMeWeekly” Scheduled Task name. In this example this rule is triggered by two different Scheduled Tasks, one that runs every day and one that runs once a week.

Available Variables

The above example is a very simple rule that pulls all files from a root folder to a location local to your Cerberus server. Filtering on which files are matched by the Scheduled Task can be limited by taking advantage of the suite of variables available for use with a Scan A Folder scheduled task and associated rule.

For example, in the case of recursive folders and/or files, the {{SETCNT}}, or ‘Set Count’ variable, contains the number of files in the Scan A Folder action and the {{SETLAST}}, or ‘Set Last File’ can be used to trigger the action on the last file found.

The {{TARGET}} variable contains the label from the Transfer Event Target used in the Scheduled Task. This matches the label you gave your Event Target when you set it up in 'Event Targets. This variable can be useful information to include in any emails sent about scanned files when creating an event rule referencing files scanned from a remote system.

Complete list of the variables for this rule type (as of Cerberus 2024.2.0):

Using Time Based Conditions

There are 2 broad categories into which the time based variables can be divided:

1. Absolute Date: Absolute time variables are used when scanning for files older than a specific date by making use of Date Picker on the user interface. See more details of their exact usage below.
   List of absolute time variables:
   • TCRT  (Time Created)
   • TACC (Time Accessed)
   • TMOD (Time Modified)

Examples of usage:

I) Scan for files whose creation time is before 15th April.

II) Scan for files whose modification time is after a specific date-time.

2. Filter Focused: As the name indicates, these are time variables used when scanning older files relative to current time. For example, they could be used to scan for files which are 3 weeks older with respect to current time. These provide greater control to the users by making use of time expressions. More details on their exact usage is below.
   List of absolute time variables:
   • {{TCRTSTR}}    - Time Created Structure
   • {{TACCSTR}}    - Time Accessed Structure
   • {{TMODSTR}}  - Time Modified Structure

Examples of usage:

I) Scan for all files which were created before the last hour.

II) Scan for all files modified 1 year past their creation time.

There are relative time variables which represent the most commonly used time periods. These variables are used to scan older files using relative time by using simple time expressions. The variables are listed below:

• {{LAST1HR}}     - One hour in seconds
• {{LAST24HR}}   - 24 hours in seconds
• {{LAST7D}}        - 7 days in seconds
• {{LAST30D}}      - 30 days in seconds
• {{LAST365D}}    - 365 days in seconds

Example of usage:

I) Scan files older than 30 days(using creation date):

To take advantage of these variables, provide any simple math expression as the filter value when using relative time variables. The value provided must only contain numbers or numeric based event variables (eg. {{T}}, {{SETCNT}}, {{TMOD}}). Numeric based event variables should be entered manually in the ‘Value’ field.

Note:

i) Only addition and subtraction operations are allowed on the numerical values.

ii) Any other non-numerical values or other operations like multiplication, division etc. are not currently supported.

Examples of filter expressions:

Filter Expression Validity

{{T}}-abcd - Invalid Expression (non-numeric)

{{TASK}}-{{T}} - Invalid Expression (arithmetic operation on strings)

{{T}}%20 - Invalid Expression (HTTP encoded character)

{{T}}-{{LAST30D}}-3600 - Valid Expression (valid arithmetic formula)

The following use-cases are provided to allow for better understanding of the feature usage and the purpose of absolute & relative time variables.

I) Use case #1:  You want the rule to perform some operations only on files which existed before a given date. In this case you can make use of the absolute time variables to look for all files which were present before the defined date. Add a condition using the TCRT variable and a date. Either use the date picker or type in the date:

II) Use case #2: You want to run a scheduled task which will scan all files which have not been modified for more than 1 year (365 days). In this case, make use of relative time variables in a simple math expression to find such older files. Either of the two condition types can be specified to scan for such files:

Condition using the predefined 'LAST365D' variable:

Alternative Condition using number of seconds in 365 days:

Note: LAST365D represents 31536000 seconds (number of seconds in 365 days) so they can be used interchangeably.

Once you have defined the rule conditions, in the 'Action' section, you can define what you would like this rule to do. See Creating and Editing Event Rules for more details.