How to Handle External Feeds for File Updates

In IT environments, it is sometimes necessary to control the flow of network traffic based on exterior information sources, i.e., accessibility of of specific networks, domains, and/or IP addresses.

To use such information on a CloudGen Firewall, networks and IP addresses can be imported on a Control Center from such exterior sources into network objects via different file types, i.e. CSV, and others.

As an example, typical sources can be providers like blocklist.de, spamhaus.org, hunt.io, and others.

Depending on the individual requirements and the configuration, the Control Center can pass on information from the downloaded external feeds to manages boxes via the Global Network Objects based on the associated box configurations.

Note that any information from an external source should be verified before being used!

Uncritical usage of unverified information (domains, IP addresses) can lead to unexpected effects and can even render harmless network portions unreachable if these sites are being referenced by domains/IP addresses considered to be part of a block list!

External Feeds, Scope and Limitations

The External Feeds feature is an implementation for firmware 10.5 and was created to be used on the Control Center. At its core, it is a list of IP and network addresses provided by an external source to be used by the Control Center. Operationally, this list is represented by a URL, which tells from where the list can be downloaded to the CC. The configuration scale on the CC is global, not per range, cluster, or box.

Because different appliances have different sizes of memory, and because the lists can be numerous and very long, you can limit the number of lines being imported from an external feed. Note that any line that exceeds the limit will not be imported. To keep these lists updated regularly, you can also enter the interval after which the Control Center will refresh the list’s content.

The download can be done via 3 options (Direct, System-like, Proxy), which you can select during the configuration.

For the feed itself, you must provide the URL where the CC can find the required data. Disabled feeds will not be continuously updated, but the firewall object with its last content can still be referenced. This is helpful if you have a specific version of data which you trust and don’t want to be updated regularly.

You can also specify which HTTP header(s) must be sent for fetching the external feed from the URL. The preset value for the feed format indicates in whether the feed data shall be handled in CSV or JSON format.

If CSV is selected, you have multiple options how the data has to be interpreted to extract the relevant information from the CSV file.

How to Handle External Feeds for File Updates

To configure an external feed, perform the following steps on a Control Center:

Go to CONTROL > File Updates.
From the drop-down menu list below the ribbon bar, select External Feeds.
Click Set Area Config… to invoke the configuration window.

NOTE: Up to 4 feeds can be configured!

If you want to rename an existing feed, be aware that renaming the entry will delete all contents in the affected feed. However, the references from the global object to the former, empty feed object will remain!

The External Feeds configuration windows is displayed.
To create a new external feed, click the green '+' on top of the section External Feeds.
The External Feeds window is displayed.
For Name, enter the name for the new external feed.

Renaming an external feed after its first creation will delete the old Global Firewall Object and create a new one.

Click OK… .

The configuration window for external feeds is displayed.
Enter the full qualified path for the External Feed URL.
Align the preset value for Connection Timeout (sec) if necessary.
If you want to add specific information for HTTP Headers, click the green '+' above the related section.
Option #1: If you select CSV for Feed Format
1. For CSV Delimiter, enter the delimiting character that indicates the boundary between two columns.
2. For CSV Comment Prefix, enter the character sequence which signifies if the content of a line must be ignored.
  Example: /* will ignore all lines starting with /*.
3. For CSV Column Header, activate the check box if the first data line should be used as the header.
4. For CSV Entry Column, align the preset value if necessary.
5. For CSV Quote Character, enter the character which indicates the location (begin and end) of a special character sequence within the returned data.
Option #2: If you select JSON for Feed Format
1. Click the green '+' to add a JSON path to the list.
2. If you want to add another JSON path, repeat the previous step.
Click OK so save your input.
Max. List Entries – If necessary, change this value according to your requirements.
NOTE: This preset value (=50000) limits the number of lines the CC reads from the external feed. If the number of available lines exceeds this limit, all non-read lines will be ignored.

NOTE: You can also modify the maximum number of network entries of an object to be displayed. For more information, see Barracuda Firewall Admin Settings.
Time Settings
1. For Download Intervall, change the preset value (=30 min.) if required.
  Note that if a feed can not be parsed after a consequent reload, the previous version of the feed is kept and the new data is discarded.
2. For Concurrent Updates, change the preset value (=0 = Auto) if required. This parameter signifies the threshold for concurrent box updates for the same area.
Proxy Settings
1. For Connection Type, select the value the fits your requirement best:
  - Explicit Proxy – If you want to use a HTTP proxy server, enter the related proxy configuration data.
  - Direct – Do not use any proxy. The external feed will be downloaded directly to the Control Center.
  - Like-CC-Box-System-Settings – Take the connection data from the box' system settings.
Click OK.

Availability of External Feeds in Global Network Objects on the Control Center

Once an external feed has been imported onto the Control Center, you can check the presence of the external feed in the list of network objects.

To do so, go to CONFIGURATION > Configuration Tree > Global Settings > Firewall Objects/Policies.

In the image below, you can see where the external feed is located in the STATIC list of network objects.

Checking the Global Network Object for External Feeds on the Managed Firewall

After the Control Center has downloaded the external feeds and pushed the data through the Global Firewall Object to its managed firewalls, you can check the feeds content by performing the following steps on the managed firewall:

Log into the managed firewall.
Go to FIREWALL > Forwarding Rules > Networks and expand the node SHARED in the list view.
Locate the related entry of the Global Firewall Object and double-click on it. The name of the Global Firewall Object must be the same as the name of the external feed on the CC.
The Edit/Create Network Object window is displayed.
In the list view Include Entries, you can see the IP addresses of the Gobal Network Object.

Deleting an External Feed

Before deleting an external feed, make sure to remove any references to its Global Firewall Object!

To delete an external feed, perform the following steps:

Go to CONTROL > File Updates.
From the drop-down menu list below the ribbon bar, select External Feeds.
Click Set Area Config… to invoke the configuration window.
The External Feeds configuration windows is displayed.
In the list view of the section External Feeds, select the feed to delete.
Click the red 'X' to permanently delete the external feed.

Renaming an External Feed

If you want to rename an External Feed, note that renaming will delete the old Global Firewall Object and create a new one!

To rename an External Feed, perform the following steps:

Go to CONTROL > File Updates.
From the drop-down menu list below the ribbon bar, select External Feeds.
Click Set Area Config… to invoke the configuration window.
The External Feeds configuration windows is displayed.
Right-click the entry in the list of External Feeds which you want to rename.
The windows for renaming the feed is displayed.
Enter the new name for the external feed.
Click OK.