DELL EMC ISILON SMARTLOCK COMPLIANCE

MODE BEST PRACTICES

ABSTRACT
This white paper explains the Isilon SmartLock Write Once Read Many (WORM)
software features including best practices for deployment in compliance and regulatory
environments.

October 2016

WHITE PAPER
The information in this publication is provided as is. DELL EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular
purpose.

Use, copying, and distribution of any DELL EMC software described in this publication requires an applicable software license.

DELL EMC2, DELL EMC, the DELL EMC logo are registered trademarks or trademarks of DELL EMC Corporation in the United States
and other countries. All other trademarks used herein are the property of their respective owners. Copyright 2016 DELL EMC
Corporation. All rights reserved. Published in the USA. <10/16> <white paper> <H13867>

DELL EMC believes the information in this document is accurate as of its publication date. The information is subject to change without
notice.

DELL EMC is now part of the Dell group of companies.

2
TABLE OF CONTENTS

EXECUTIVE SUMMARY ...........................................................................................................4

AUDIENCE .................................................................................................................................4
CLUSTER MODES ....................................................................................................................4
COMPLIANCE CLOCK .............................................................................................................4
ENTERPRISE MODE.................................................................................................................4
ENTERPRISE DIRECTORIES .......................................................................................................... 4

COMPLIANCE MODE ...............................................................................................................5

COMPLIANCE DIRECTORIES ......................................................................................................... 5
Removal of root access ..................................................................................................................... 5
SmartLock Directories Feature Comparison ..................................................................................... 6
Compliance mode usage best practices ......................................................................................... 6
SyncIQ with compliance mode .......................................................................................................... 7

FREQUENTLY ASKED QUESTIONS (FAQS) ..........................................................................7

APPENDIX A .............................................................................................................................9
APPENDIX B .......................................................................................................................... 12

3
EXECUTIVE SUMMARY
Dell EMC Isilon SmartLock is a licensed software module available with Isilon OneFS 6.5.5 onwards. It is used to protect critical
data from unauthorized alteration. SmartLock allows you to commit files to a Write Once Read Many (WORM) state, which prevents
users from erasing or re-writing those files.

AUDIENCE
This white paper is intended for System Engineers, Storage Administrators, Security Managers, and IT Managers that need an
overview of Isilon SmartLock and the best practices around deploying and implementing SmartLock in a regulatory environment

CLUSTER MODES
An Isilon cluster can operate in one of the following modes:

Standard or Normal mode: This is the default cluster operational mode if the SmartLock license is not purchased or activated.
This is not a SmartLock mode.

SmartLock Enterprise mode: If the SmartLock license is activated, this becomes the default cluster operational mode. An
Enterprise mode cluster permits implementation of Enterprise SmartLock directories and committing data to a WORM state for
a specified data retention period.

SmartLock Compliance mode: If the SmartLock license is activated, the cluster can optionally be put into Compliance mode. In
this mode, it is possible to protect data in compliance with the regulations defined by U.S. Securities and Exchange
Commission rule 17a-4 by creating SmartLock compliance directories.

Note: The mode of operation is cluster wide.

COMPLIANCE CLOCK
In the context of SmartLock, there are 2 types of clocks System clock and Compliance clock.

System Clock is the standard clock that is common to both Enterprise and Compliance modes. Compliance clock is exclusive to
Compliance mode. The compliance clock updates the time in a protected system B-tree entry. Unlike the System clock, the Compliance
clock cannot be manually modified by the root or compadmin user, which could lead to the files being released from a WORM state
earlier than intended.

The cmd: isi worm cdate set can be used to set the WORM compliance clock.

The cmd: isi worm cdate view can be used to view the WORM compliance clock.

ENTERPRISE MODE
Enterprise mode permits storing data in non-rewriteable, non-erasable format, protecting data from deletion or modification.

ENTERPRISE DIRECTORIES
Enterprise directories contain data in a non-rewritable, non-erasable format. Enterprise directories can be created in both Enterprise as
well as Compliance modes. If a file in an Enterprise directory is committed to a WORM state, it is protected from accidental deletion or
modification until the retention period has expired.

In Enterprise mode, regular directories may also be created. Regular directories are not subjected to retention requirements.

A cluster operating in SmartLock Enterprise mode provides advanced security capabilities while retaining superuser root access and
full administrative control. In the majority of situations, Enterprise mode offers more than adequate security requirements for most
users.
4
COMPLIANCE MODE
SmartLock Compliance mode is designed only for users who are required to preserve critical electronic records in order to comply with
the United States Securities and Exchange Commissions (SEC) rule 17-a4(f), relating to the electronic storage of broker-dealer
records. The level of security required by rule 17-a4(f) is so stringent that not even administrators should be allowed to modify or delete
compliance WORM data.

COMPLIANCE DIRECTORIES
In Compliance mode compliance directories are created for WORM data that must be protected in compliance with U.S. Securities and
Exchange Commission rule 17a-4. Compliance directories are governed by the compliance clock. As mentioned earlier, the
compliance clock cannot be modified.

Following table shows what type of directories and files (data) can be created in each of the cluster modes.

Enterprise Mode Compliance Mode

Regular (non-SmartLock) Directories Yes Yes
Enterprise Directories (governed by system Yes Yes
clock)
Compliance Directories (governed by No Yes
compliance clock)

Note: Both SmartLock cluster modes (Enterprise and Compliance) also support the creation of standard or regular directories and files
that are not subjected to retention requirements.

Removal of root access

Operating in Compliance mode will disable root (superuser) access to the cluster in all circumstances. Superusers (UserID 0) will be
unable to login by any means, including in single-user mode. Instead of the root user, clusters operating in compliance mode have an
administrator account called "compadmin". This account provides allows administrators to run some commands with root privileges
through sudo. These commands are specified in /usr/local/etc/sudoers file

Appendix A shows the list of commands that can be run through sudo by compadmin in OneFS version 7.2.0

In addition to the list described in Appendix A, the compadmin user can execute any isi commands that belong to modules that are
Role-Based Access Control (RBAC) enabled. These are:

isi auth
isi zone
isi smb
isi job
isi nfs
isi snapshot
isi quota
isi sync
Appendix B shows the list of operations that cannot be performed once a cluster is put in Compliance Mode.

The following table summarizes the differences between features for Enterprise and Compliance SmartLock directories:

5
SmartLock Directories Feature Comparison
Feature Enterprise Directories Compliance Directories
Customizable File Retention dates Yes Yes
Protection from modification after commit Yes Yes
SEC 17a-4(f)-compliant file retention No Yes
Privileged delete On|Off|Disabled * Disabled
Tamper proof compliance clock Not Available Yes
Superuser (root) account Yes Not Available
Sudo based cluster admin account (compadmin) Not Available Yes

Note* - In Enterprise mode, privileged delete remains available and configurable. It is Off by default and can be turned On for
Enterprise directories. It can also be permanently disabled for enterprise directories thus protecting data from deletion or modification
under all circumstances. In Compliance Mode it is disabled by default for Compliance directories.

Compliance mode usage best practices

The administrative restrictions of Compliance mode have the potential to affect both compliance data as well as enterprise data. In
order to make an informed decision concerning SmartLock, Isilon offers the following guidelines and suggested best practices:

Isilon recommends implementing Compliance mode only if your organization is legally obligated to do so under SEC rule 17-a4(f).
As the Compliance mode installation or upgrade involves careful planning and preparation, it is recommended to be done with the
assistance of Isilon Support.
Enterprise mode offers advanced security capabilities, such as the protection of directories and files from deletion in a WORM
state, the ability to disable privileged delete, and overall security. Therefore, with regard to safeguarding data, Enterprise mode
offers more than adequate security requirements for most users, in the majority of situations. Moreover, the superuser account
remains available in Enterprise mode. Therefore, it is more administrator friendly compared to Compliance mode.
Following are the best practices that need to be performed prior to putting an existing cluster in Compliance mode.

Test and validate all workflows using a proof-of-concept Compliance mode cluster. Use Isilon OneFS Simulator as a Virtual
Machine (VM) test host, if available.
Verify that the cluster time is correct before putting the cluster in Compliance Mode.
Do not use run-as-root on SMB shares. If you have previously configured SMB shares to run-as-root then change the settings for
those shares to specify access permissions to either Full-Control, Read-Write or Read before putting the cluster in compliance
mode.
Use Role based access control (RBAC) for cluster access to execute file management and administrative operations. Enable
RBAC, grant appropriate privileges, and then log on through the RBAC-enabled account to the command line interface (CLI).
compadmin represents a regular data user in the context of the CLI.
For any data migrations from a non-Compliance mode cluster to a cluster that you intend to put in compliance mode, first verify that
current ownership and access permissions are valid and appropriate on both clusters so as to allow data migration.
Review the permissions and ownership of any files that exclusively permit the root account to manage or write data to them. Once
an upgrade to Compliance mode is complete, if the existing OneFS configuration limits relevant POSIX access permissions to
specific directories or files in any way, writing data or changing ownership of these objects will be blocked.
If any root-owned workflow or data files exist, all ownership or permission changes should be executed before upgrading to
Compliance mode. You should not change the ownership of any system files. The Compliance mode conversion process
automates all required ownership changes to system files. You should not change the ownership of any files outside of /ifs, as no
user data should reside outside of /ifs. As a best practice, change the ownership of files under /ifs that are owned by root to the
compadmin account before upgrading to Compliance mode.
In Compliance mode, the default POSIX permissions permit the compadmin account to write data. However, the following
directories should not be modified unless the default permissions for these directories have been changed : /ifs/.ifsvar and
/ifs/.snapshot
Verify the available disaster recovery options on compliance mode clusters in relation to SyncIQ. Refer to the section SyncIQ with
Compliance Mode for more details.

6
 Note that if NDMP is being used for backups, the NDMP backups of SmartLock Compliance data are not considered to be
compliant with the SEC regulation 17-a4f.

The following table summarizes the difference between the SmartLock modes.
SmartLock Mode: Enterprise SmartLock Mode: Compliance
Governed by a single clock System Clock. Governed by 2 clocks System clock and Compliance
Clock.
Data written to Enterprise SmartLock directories is Data written to Compliance SmartLock directories once
committed to WORM state only for the specified retention committed can never be altered.
period.
Superuser access (root access) is maintained with full Superuser access is disabled.
administrative control

SyncIQ with compliance mode

SyncIQ can be used with SmartLock to replicate WORM-protected data sets. The following table summarizes the compatibility of
SyncIQ replication actions based on the source and target directory types.

Source Directory Type Target Directory Type Replication Allowed or Not

Standard Non-WORM Standard Non-WORM Yes
Standard Non-WORM Enterprise SmartLock Yes
Standard Non-WORM Compliance SmartLock No

Enterprise SmartLock Standard Non-WORM Yes

Enterprise SmartLock Enterprise SmartLock Yes
Enterprise SmartLock Compliance SmartLock No

Compliance SmartLock Standard Non-WORM No

Compliance SmartLock Enterprise SmartLock No
Compliance SmartLock Compliance SmartLock Yes

SyncIQ replicates the data asynchronously from the Primary Isilon cluster to one or many clusters. This creates multiple copies of data.
This can be used in case of a failure on the primary Isilon cluster. The process of activating a secondary copy for read-write purposes is
called Failover. This is usually done when the primary Isilon cluster becomes unavailable or is taken down for maintenance. When the
original primary cluster is back online, a reverse sync can be established using SyncIQ. If required, the original primary can be made
the read-write copy. This process is called Failback.
Starting with OneFS 8.0.1 SyncIQ failover and failback can be done on Compliance Mode SmartLock directories, delivering automated
disaster recovery for financial services SEC-17a4 regulatory compliance.

Frequently Asked Questions (FAQs)

The following table lists some of the frequently asked questions and their answers.

7
 Are there any necessary pre-upgrade tasks or health checks to carry out before putting a cluster into
Q Compliance mode?
A pre-requisite for upgrade to compliance mode is that all nodes must be up and running. Otherwise,
A the upgrade will not succeed.
What pre-upgrade workflow-specific administrative tasks must be done? What about custom
Q configurations?
All existing administrative custom configuration that requires superuser root access must either
be undone or, alternatively, ownership must be given to the compadmin account. For example: all
custom mount points created by the user should be removed from /etc/fstab before putting the cluster
A in Compliance mode.
Q What file ownership or permission changes need to be done, if any?
All custom-created, root-owned directories that are writeable only by the root account should be
removed. Alternatively, ownership and permissions should be transferred to the compadmin account
A by using chown and chmod commands.
If there are existing log and crash directories, how to ensure that one can still write to those directories
after upgrade to compliance mode?
Q
You only need to ensure that such directories have the necessary permissions to perform the required
actions before upgrading to compliance mode.
A
Q Can the compadmin account take ownership of root-owned files, post-upgrade?
No. You cannot change or take ownership of root-created or root-owned files, post-upgrade. If
permissions are set to deny this, sudo chown and chmod will fail.
A
What about NFS configuration using root? Can the root user be mapped to "nobody" or to any root
Q mappings?
Isilon does not take specific action to prevent this, because the user is not logged in as root, and so
cannot execute actions that could potentially damage data. Compliance mode prevents all users who
employ standard POSIX commands from writing to or deleting committed files.
A
Q What happens to SMB shares configured to run-as-root? Do they need to be modified pre-upgrade?
A Yes, because run-as-root does not function correctly in compliance mode.
Q Are there any known Job Engine dependencies? Are maintenance jobs affected by compliance mode?
You can run all Job Engine jobs as usual after upgrade. All Job Engine jobs still run as root, having
root privileges. The operation of jobs remains unaffected.
A
Q What is the full list of sudo commands available from compliance mode?
A You may view /usr/local/etc/sudoers on any cluster installation.
Q Can existing enterprise SmartLock directories be "upgraded" to compliance mode?
Yes, on condition that the existing enterprise directories are empty. This is one of the typical paths
used to create a compliance SmartLock directory.
A
Q Does upgrade to compliance mode impact existing enterprise SmartLock directories or configuration?
A No, not unless root-based workflows were not previously remediated.
Q Are there any issues when adding nodes, post-upgrade?
No known issues. A node must be converted to compliance mode before it can join a compliance
A cluster.
What happens if a node goes down during the upgrade because of a hardware failure, such as a drive
failure, failed boot drive, or other node-impacting issue?
Q
If a node goes down during the process, the upgrade is aborted and rolled back on all functioning
nodes. The node may need to be Smart-failed. Alternatively, the issue may need to be addressed by
A Isilon support.
Q If a node fails, can it be removed, post-upgrade?
A Yes, you can SmartFail a disk or node and remove it.
Q What if I have custom configurations on my cluster?
The upgrade has the potential to impact any specific root-created or root-managed custom
configurations. Therefore, you should either modify or remove such configurations before upgrade.
A
8
 Q Are there any known snapshot issues, pre or post-upgrade?
A No known issues.
Q Are there any known Quotas issues, pre or post-upgrade?
A No known issues.
Q What about Rollback? Is reimaging the only available method to return to enterprise mode?
Yes. If a failure occurs during upgrade, the cluster will try to roll back any changes, but successful
rollback is not guaranteed, especially in the case of a failed node. After successful upgrade, no
rollback is possible. In that case, only a full re-image can change the cluster back to enterprise mode,
A possible only by using a boot USB flash drive.
Q Does upgrading to compliance mode have any known impact on sysctls?
A None.
Q Does upgrading to compliance mode have any impact on patches? Should I remove or leave them?
A No known issues. No action required.
Q Are there any issues when adding an accelerator node to an existing compliance cluster?
A No known issues. All known issues have been resolved.

Appendix A
Following commands can be run by compadmin user using sudo when a cluster is in Compliance Mode. This can also be found in the
file /usr/local/etc/sudoers.

9
Category Commands
Processes /bin/kill, /usr/bin/renice
/usr/bin/pkill, /usr/bin/top, /usr/bin/killall

System Administration /bin/date, /sbin/sysctl, /sbin/shutdown

/bin/ps, /usr/sbin/ntpdate, /sbin/ifconfig
/usr/sbin/nfsstat, /usr/sbin/pciconf
/usr/sbin/tcpdump

Cluster Administration /usr/bin/isi

/usr/bin/isi_gconfig, /usr/bin/isi_job_d
/usr/bin/isi_for_array, /usr/bin/isi_vol_copy

Support /usr/bin/gcore, /usr/bin/isi_bootdisk_status

/usr/bin/isi_client_stats, /usr/bin/isi_cpr
/usr/bin/isi_dmi_info, /usr/bin/isi_dmilog
/usr/bin/isi_dongle_sync
/usr/bin/isi_drivenum
/usr/bin/isi_dsp_install
/usr/bin/isi_dumpjournal
/usr/bin/isi_eth_mixer_d
/usr/bin/isi_evaluate_provision_drive
/usr/bin/isi_flush
/usr/bin/isi_gather_auth_info
/usr/bin/isi_gather_cluster_info
/usr/bin/isi_gather_info
/usr/bin/isi_get_itrace
/usr/bin/isi_get_profile
/usr/bin/isi_hangdump, /usr/bin/isi_hw_status
/usr/bin/isi_ib_bug_info, /usr/bin/isi_ilog
/usr/bin/isi_imdd_status
/usr/bin/isi_inventory_tool
/usr/bin/isi_km_diag
/usr/bin/isi_linmap_mod
/usr/bin/isi_make_abr, /usr/bin/isi_nodes
/usr/bin/isi_ntp_config
/usr/bin/isi_ovt_check, /usr/bin/isi_patch_d
/usr/bin/isi_promptsupport
/usr/bin/isi_radish
/usr/bin/isi_rbm_ping
/usr/bin/isi_repstate_mod
/usr/bin/isi_restill
/usr/bin/isi_save_itrace
/usr/bin/isi_send_abr
/usr/bin/isi_smbios
10
 /usr/bin/isi_stats_tool
/usr/bin/isi_transform_tool
/usr/bin/isi_ufp
/usr/bin/isi_umount_ifs
/usr/bin/isi_update_cto
/usr/bin/isi_vol_copy
/usr/bin/isi_vol_copy_vnx
/usr/bin/nfsstat
/usr/sbin/isi_bootdisk_finish
/usr/sbin/isi_bootdisk_provider_dev
/usr/sbin/isi_bootdisk_unlock
/usr/sbin/isi_checkjournal
/usr/sbin/isi_disk_firmware_reboot
/usr/sbin/isi_clean_idmap
/usr/sbin/isi_fsa_tool
/usr/sbin/isi_kill_busy
/usr/sbin/isi_lid_d
/usr/sbin/isi_logstore
/usr/sbin/isi_mcp
/usr/sbin/isi_netlogger
/usr/sbin/isi_savecore

Hardware Tools /usr/bin/isi_hwtools/isi_cdesexputil

/usr/bin/isi_hwtools/isi_cto_convert
/usr/bin/isi_hwtools/isi_cto_update
/usr/bin/isi_hwtools/isi_cto_upgrade
/usr/bin/isi_hwtools/isi_fcb_vpd_tool
/usr/bin/isi_hwtools/isi_fputil
/usr/bin/isi_hwtools/isi_flexnet_info
/usr/bin/isi_hwtools/isi_hw_check
/usr/bin/isi_hwtools/isi_ib_fw
/usr/bin/isi_hwtools/isi_ib_info
/usr/bin/isi_hwtools/isi_ipmicmc
/usr/bin/isi_hwtools/isi_lsiexputil
/usr/bin/isi_hwtools/isi_mpsutil
/usr/bin/isi_hwtools/isi_mps_fw_status
/usr/bin/isi_hwtools/isi_mpt_fw_status
/usr/bin/isi_hwtools/isi_ovt_check
/usr/bin/isi_hwtools/isi_rnvutil
/usr/bin/isi_hwtools/isi_sasphymon
/usr/bin/isi_hwtools/isi_sed
/usr/bin/isi_hwtools/isi_update_serialno
/usr/bin/isi_hwtools/isi_vitutil
/sbin/nvmecontrol

11
Appendix B
Operations that cannot be performed in Compliance Mode are:
The root account cannot be used after a cluster is put in Compliance Mode.
Files that are owned by root cannot be modified using combination of sudo and compadmin after the cluster is put in Compliance
Mode.
A SmartLock directory cannot contain another SmartLock root directory. This is applicable for both Enterprise SmartLock
directories and Compliance SmartLock directories.
A directory cannot be set as a compliance or enterprise SmartLock directory if it already has files/directories under it. In other
words, only an empty directory can be set to compliance SmartLock directory.
Hard links cannot cross SmartLock directory boundaries.
Writing to directory that has not finished converting to a SmartLock directory is allowed, but the files cannot be committed until the
SmartLock directory is ready.
In Compliance mode, if there is an existing Enterprise SmartLock directory, it can be upgraded to a Compliance SmartLock
directory. However, the change is one-way only. A compliance SmartLock directory cannot be reverted to an Enterprise SmartLock
directory.
If the compliance clock has not been set on the cluster, attempting to upgrade a directory to SmartLock Compliance directory will
fail.

12