Can Cerberus download multiple files from a remote server?

As of Cerberus 2024.2.0, an enhancement to the 'Scan A Folder' action of Scheduled Tasks allows a user to scan and manipulate every file on a remote FTP or FTPS server without needing to know the specific files in the folder. The enhancement allows the Cerberus admin to set up a Scheduled Task that uses an FTP or FTPS Transfer File Target to connect to a remote server via FTP and FTPS to scan a remote folder and gather the filenames of all files in the folder. Additional SFTP functionality will be added to a future Cerberus.

Each file found in the scanned folder will trigger the associated File Scanned Event rule you configure in Event Rules.  The Event Rule can perform operations on the files in the folder, such as downloading them. With this event’s ability to specify a condition on the generating task’s name, the rule can be restricted to only occur for a specific Scheduled Task.

Cerberus FTP Server 2024.2.0 will also now allow you to filter files based on their age (creation, modification, or access time) on your local system as well as on remote FTP and FTPS servers. This functionality expands the existing "Scan a Folder" scheduled task action and "File Scanned Events" rules.

Key Enhancements:

• Remote File Filtering: Apply age-based filters to files on remote FTP and FTPS servers.
• Extended Wildcards: Utilize standard Windows wildcards (* and ?) for filename matching during remote folder scans.
• New Event Variables: Access new variables (TCRT: Time Created, TACC: Time Accessed, TMOD: Time Modified for filtering in event rules.
• Additional Event Targets for Actions:  FTP and FTPS folders (using predefined Transfer File Targets).

COMING SOON: Support for scanning a remote SFTP server.

Example Scan A Folder workflow to a remote server

These instructions give an example as to how a Scan A Folder workflow to a remote server might occur. This example has a scheduled task that triggers every night at midnight. The Scan A Folder action looks at a folder on a remote server and, for each file there, sends out a File Scanned Event. These events are processed in order by the Events System, each time looking for rules that trigger off of the given File Scanned Events.  In this example, the rule downloads all the files to a folder on the local Cerberus server.

Defining the Scheduled Task

The Scan A Folder workflow is broken into two steps. The first step to using this feature is to define when this action is to occur. Navigate to Event Manager > Scheduled Tasks and click the 'New' button to add a new task. 

Define a time for this task to occur, and how often it will repeat.  In this example case, the action will occur daily at midnight.

Create a new action by using the new button and choosing Scan A Folder in the Action: drop-down menu.

This action has the same top elements as all other actions: An action type and ‘using’ field gets filled in for Scan A Folder and is not editable and the standard “Stop on Failure” checkbox as all actions have, which determines if processing will continue to the next action if this action fails.

Scan A Folder Scheduled Task Actions are now able to use predefined FTP and FTPS transfer file targets, set up in the same way as Event Rule Event Targets (See Configuring Event Targets), to scan for files on remote FTP and FTPS servers. Support for SFTP is planned for a future Cerberus release.

Using the Scan a Folder targets:

"FTP File Transfer Target": Scan a folder on a remote FTP Server

"FTPS File Transfer Target": Scan a folder on a remote FTPS Server

Below is an example of an "FTP File Transfer Target" set up in 'Event Targets' with an IP address of 127.0.0.1 and FTP protocol defined in the event targets list.

We can use this event target in our scan a folder scheduled task. This scheduled task uses this target to log on to a remote FTP server and list all the files in the user's root directory. This basically will be a scan of ftp://load@127.0.0.1/*.*.

In this action, “Using:” is set to the FTP file transfer target.

Below 'Using' is the "Fail If:" subsection. This section helps define what Scan A Folder should consider a terminal failure state, and when it will ignore and continue to process the next action. Failure options include:

• Fail if the action is Unable to Read Every File in the given folder
• Fail even if the action is only Unable to Read Some Files
• Fail if Zero Files Selected by Filter (in other words, files are found, but none that qualify for processing).
• Fail if any File Scanned Event Rule Actions Fail (where one of the event rules that handled the new File Scanned Event returned an error)

These options all work together to determine if this action failed in terms of the prior checkbox.

“Scan Folder Path” is the directory you’d like to scan.  In this example, since the task is scanning a remote location, the field can either be left empty to scan the user's root directory, or you can enter a relative remote path. Cerberus FTP Server will create a File Scanned Event for each file found in the scanned path.  If you’d like not only the defined path, but all of its children, simply check the “Recursively transfer all sub-folder content” box.

The filter string is a way to pick up only a single type of file, or a partial filename, if desired.  It will append the filter string to the folder path if it’s in the form of * for a wildcard, and ? for single characters.  For example, *.txt will create events for every text file in the directory.

In the example above, the folder path is left blank to search the remote root folder with a filter string set to search for all text format files using a wildcard * in the filter string. An empty Filter String field scans all files. This creates a list of four File Scanned Events, one for each of the files in the folder.

Defining the Event Rule

Next, you will create a rule that will be triggered off of the Scheduled Task.  Navigate to Event Manager > Event Rules and click 'New' to add a new rule.  Then, select a File Scanned Event rule type.

To tell Cerberus which rule to trigger when the Scheduled Task runs, in the Rule Conditions, use the {{TASK}} variable '==' (equals) the name of the "FolderScan" Scheduled Task you set up above.

In this example this rule is triggered when the Scheduled Task name is 'RunMeDaily'.

Note that the 'Match These Conditions' setting can also be set to match more than one task name. For example, if you want the rule to be able to be triggered both by a “RunMeDaily” Scheduled Task, and a “RunMeWeekly” Scheduled Task name, set 'Match These Conditions' to Match if ANY Filters Match. 

Next you will configure the Rule Action. In this example, we use the FTP Event Target to download all files found on the remote folder and pull them down to a local folder on the Cerberus server:

In the 'File Path From', when using a File Transfer Target, use the {{RFP}} the remote (relative) file path variable to pass the path and file name of each file to retrieve to the rule. The 'File Path To' should contain the path to the local folder on the Cerberus server you wish to place the files into and then use the {{LFN}} (File Name) variable to pass the name of each file. BOTH your FROM and TO paths MUST include a way for Cerberus to write the filename. DO NOT use wildcards here. The rule is triggered once per filename and each filename must be passed in the action.