ONTAP File System Analytics: Better visibility for better management

Data has become the lifeblood of organizations, and there is an unprecedented growth in unstructured data. Organizations are looking at monetizing the value of data so that they can manage the infrastructure more effectively and base calculative decisions on the data rather than on hunch.

No insights, then no direction

Imagine being at a crossroad and having no clue about the right direction to take to reach your destination. It's hard to imagine a road trip without Google Maps unless you have one path to take.

File System Analytics (FSA), introduced in NetApp^® ONTAP^® 9.8, helps you navigate through data and gives you better visibility to manage your data effectively. Perhaps FSA won't help you find your life's purpose, but it can help you find your assets' value through data. FSA helps solve organizational data challenges around data locality, ownership, storage consumption, storage optimization, and data protection.

Discovery: How ONTAP File System Analytics collects the metadata and visualizes

FSA is a native engine that discovers file metadata automatically and doesn't need any external tools. To discover the metadata, FSA uses two methods:

• For an existing dataset: FSA runs a scanning process to collect and organize the information about the file system.
• For incoming files: Metadata is collected while the data is being written to the volume.

Integrations and API

FSA collects and aggregates metadata at the directory level and presents it through a REST API interface in real time. ONTAP System Manager uses the REST API and visualizes the insight.

FSA is designed to support multiple use cases. ONTAP 9.8 provides the following ones:

• Data discovery
• Capacity usage
• File and directory counts
• File activity trends
• Active and inactive data identification

Other analytics solutions can leverage the REST API to collect the analytics information from the storage system and provide extended use cases such as capacity trending and storage reporting.

File System Analytics through ONTAP System Manager

ONTAP System Manager provides the visual interface for FSA. After you select Storage > Volumes, you'll see the new Explorer tab. This tab helps you navigate through different options to enable or disable file analytics and to see details collected about files and directories.

The Explorer tab has two views: list view and graphical view.

List view is a directory tree view of files. This view allows you to navigate to the subdirectories by expanding the parent directory, as in Windows File Explorer.

In list view, you can see how much space each directory uses; you can also see each directory's modification and access history.

You can sort the Used, Modify History, and Access History columns by clicking the column headers.

The two history options provide interactive graphs. By hovering the mouse over the trend line, you can see details about how much data has been modified or accessed in that timeline.

The trending line helps IT administrators identify where the most activity is occurring or where the data is heavily modified or accessed.

Graphical view uses variable size blocks and color shades to provide a graphical representation of the directories.

In the graphical view, you can expand to the subdirectory by clicking the parent directory. Hovering over the directory block provides a summary of directory size and subdirectory count. The color of the block depends on how inactive the directory is. Darker colors indicate that the directory was recently active, and the lightest color indicates that the directory hasn't been touched in more than a year.

Other views

Depending on the directory you select in the list view or graphical view, the Explorer tab shows a file list, active/inactive data, and the directory and file count corresponding to the directory selected.

File list. This list lets you show or hide a file's access time and size.

You can also sort by size or access time to find the largest or smallest files, or the most and least accessed files, in the root of the directory selected in the list view or graphical view.

File summary view. This view gives you an idea of how many files and directories - and how much data - have been accessed in the last week, beyond 1 week, and within 1 year. It also identifies data that hasn't been touched for 1 year.

Information from the activity trend lines (modified history and access history) in the list view and active/inactive data from the file summary view help you decide whether the data can be archived or deleted if it's obsolete. In this way, you can optimize the primary storage space and better utilize it for critical application storage.

Where to go from here

This blog post covers the use cases provided through System Manager, but there are endless possibilities for use cases with FSA data. Using the ONTAP REST API, you can collect data and store it in a time-series database. You can then visualize data growth trends or get a quick view of how storage was a year ago by comparing the status between dates. You can also compare the daily directory activity to understand the regular patterns versus a sudden spike in activity. A sudden spike can represent legitimate access by a user, or it could be malicious activity.

To get started exploring File System Analytics, take a look at the ONTAP 9.8 documentation, or view this ONTAP File System Analytics video.