Introduction to Secure File Transfer over HTTPS using Oauth2 Client Credentials Grant and Optional Encryption
Data from HR or other personnel-tracking systems are essential to be synchronized with ThreatSwitch. Employee data such as department, job title, or termination date change frequently, and would impose a heavy burden of manual synchronization.
Fortunately, ThreatSwitch’s API allows for this data to be automatically imported on a recurring basis set at the customer’s preferences. For customers using the most common ERPs or HRIS including Workday, SuccessFactors, or PeopleSoft, selective updates can be sent via an HTTPS request to ThreatSwitch. With most HR systems, export jobs can be scheduled on a recurring basis, and then routed to ThreatSwitch by a scheduled worker.
As part of the initial implementation, data fields will be mapped from the customer’s schema to ThreatSwitch’s database. The selected fields will then be automatically used and mapped for all recurring imports.
At a high-level, a typical integration flow may include the following steps:
A set of client credentials are generated by ThreatSwitch for a transactional Integration User that include a Client ID and Client Secret. These are equivalent to a username and password and should be stored securely by the customer.
Data is exported internally in a flat file format from the HRIS via SFTP to an internal location or data warehouse.
The scheduled worker uses the stored client credentials (from step 1) to authenticate with ThreatSwitch.
If successful, ThreatSwitch returns a one time use authentication token to be used in a subsequent request.
The worker uses the authentication token to authorize an HTTPS request that sends the encrypted file contents and associated metadata (including optional header mappings and/or an encryption_key_id) to ThreatSwitch.
Pending another round of security checks, the file is decrypted and imported into ThreatSwitch. All data and changes are logged as part of the import process.
Upon completion, an HTTPS response will indicate success or failure and the results of the import will be viewable within the ThreatSwitch application.
Figure 1
Authentication Overview
Per Figure 1, ThreatSwitch uses an Oauth2 Client Credential Grant to securely transfer and authenticate the data sent to it. Properly implemented OAuth2 authentications are secure, reliable, and in widespread use. The OAuth2 Client Credential Grant flow provides the basis of the OpenID Connect Single Sign On mechanism in use by ThreatSwitch customers today and is the same authentication protocol used by Amazon Web Services for all of its APIs.
Additionally, OAuth2 Client Credentials are tied to individual users within the system meaning that the permissions associated with the client credentials can be controlled and all events logged and monitored.
This authentication protocol does not require ThreatSwitch to have any knowledge of customer source data, is readily amenable to scheduled tasks, and allows ThreatSwitch to return meaningful debugging information to customers while configuring their import jobs. Compared to other implementations such as SFTP, an OAuth2-based integration method dramatically reduces the ThreatSwitch team’s direct access to customer data and ensures that our team is essentially bliind to critical PII sensitive information.
An ideal integration user for importing production data would have a Custom Role granted “Create Import” which limits permissions exclusively to allow data import and send along the relevant metadata required for decryption. The integration user can never access the decrypted data or have any direct knowledge of the associated encryption keys. Nor does ThreatSwitch or any of its employees.
Logging
ThreatSwitch’s scheduled flat file integration leverages all of ThreatSwitch’s logging and monitoring capabilities. Administrators are able to identify row-level errors in data imports, as well as track each import in real-time, simplifying debugging and long-term data cleanup.
Customer Requirements
Customers must configure a scheduled worker with stored credentials that executes the HTTPS requests. ThreatSwitch provides code samples of functioning implementations to make this process easy and ensure a short stand-up period.