Challenge: One of the largest music publishers in the United States had to maintain a complex workflow of daily corporate tasks. One such process involved tracking each instance of a song playing on radio stations and online streaming services, processing and reconciling that data, and then issuing invoices to sources and payments to artists. Each data source adhered to its own unique format, and extensive work had to be done to transform that data into a consistent format for further processing by internal applications. The company had used EFT Enterprise with active-active high availability clustering, and they did extensive automation and data validation and transformation work within the Advanced Workflow Engine to great success.
However, this process had become even more complicated with recent growth and acquisitions. The organization had acquired many smaller publishers, and they were faced with the challenge of requiring exponential growth in bandwidth demands. This would overtax the hardware on which their EFT Enterprise was managing the data.
Rather than simply expanding their on-premises implementation horizontally, adding additional nodes to handle the increased load, they opted instead to acquire and initialize a new load-balanced and highly available EFT Enterprise implementation within their growing Amazon Web Services private cloud environment. The company was able to assign each implementation a discrete set of roles in the overall corporate managed file transfer strategy, thereby managing the complex business processes. The organization needed to:
- Continue to maintain constant availability of systems,
- Process diverse formats and types of data into an expected output, and
- Reduce the strain on existing systems and scale to meet growth.
Using the AWS implementation, they were able to easily handle the growing bandwidth requirements of transfers to and from third parties, and verify that the data was well-formed, according to the expected formats. Once received, and initial basic checks passed, the files would be pushed in compressed batches from the AWS implementation to their on-premises implementation where the work of data transformation would begin, with the data then handed off to other internal application servers before any resulting invoices or statements would be sent to the AWS instance for distribution.
The music publisher was able to leverage this role-based approach to easily handle increased bandwidth demands in AWS and perform initial data checks so that bad data would never reach their internal implementation. Also, their internal on-premises implementation was able to take on the processing load in a more streamlined manner without the additional roles of servicing clients or pushing and pulling between third parties, and performing all the initial data checks without overwhelming the on-premises systems.