Music Note Transcription on Amazon ECS Fargate with TensorFlow-based Deep Learning Model

Learn how to build your personal optical music recognition (OMR) app on Amazon ECS Fargate using machine learning model

Amazon ECS Fargate Cluster running TensorFlow model for OMR

Introduction

In this blog, we will show how to use AWS CDK to deploy a Music Sheet Transcriber app on Amazon ECS Fargate powered by Streamlit UI. This app utilizes a machine learning Tensorflow model built by Calvo-Zaragoza et al. published as End-to-End Neural Optical Music Recognition of Monophonic Scores in the Applied Sciences Journal 2018.

Before we get started here are some prerequisites for this tutorial:

a. Setup AWS CDK:

If you haven’t here’s a tldr on how to do it:

1. Set up AWS CLI on your computer
2. Install Node.js, npm & Typescript
3. Install the AWS CDK with npm i -g aws-cdk
4. Bootstrap CDK with: cdk bootstrap aws://ACCOUNT-NUMBER/REGION

Get the full guide here.

b. Setup Docker:

To containerize the OMR App and prepare it for deployment, you’ll need Docker. If you don’t have Docker installed, you can download and install it from the official Docker website: Get Docker | Docker Docs.

Since we will use CDK assets ECR repository to push our docker image, make sure to login to ECR from your computer:

aws ecr get-login-password - region us-east-1 | docker login - username AWS - password-stdin YOUR-ACCOUNT-NUMBER.dkr.ecr.us-east-1.amazonaws.com

If you see the below error from CDK, it is possible you have not signed in to ECR successfully using the above command.

ERROR: failed to solve: python:3-buster: failed to authorize: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fpython%3Apull&service=registry.docker.io": EOF

Implementation

The CDK code for this solution architecture is available in our GitHub repository. To begin, you can clone the repository to your local terminal and install the necessary npm packages with these commands:

git clone https://github.com/awsdataarchitect/omr-ecs-app-cdk.git
cd omr-ecs-app-cdk
npm i

Dockerfile
The Dockerfile facilitates the deployment of a Streamlit application using python:3-buster image, incorporating key dependencies and resources:

1. Install the dependencies: tensorflow v1, Streamlit, opencv
2. Download the semantic model developed by Calvo-Zaragoza et al.
3. Download the semantic vocabulary
4. Download the font Aaargh.ttf (this is needed to annotate the image with the ABC notation)

If you would like to train the semantic model yourself, head over to the tensorflow model Github repository for instructions and download the PrIMuS dataset.

App.py (Application Code)

We will be using Streamlit to power the UI of our app. Streamlit is an open source framework to efficiently create interactive web-based data applications in pure Python. The core functionality of our app comprises of the following key steps:

• User uploads an image.
• TensorFlow model processes the image and predicts musical notations.
• Image annotations, including ABC notations, are superimposed on the original image.
• The annotated image is displayed in the Streamlit app.
• Users can download the annotated image for further use

CDK Stack
Our CDK stack uses the “aws-ecs-patterns” CDK module library that provides higher level Amazon ECS constructs to build all the above infrastructure in just 12 lines of typescript code!

const appService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'MyFargateService', {
      cluster,
      serviceName: 'ecs-omr-service',
      taskImageOptions: {
        image:   ecs.ContainerImage.fromRegistry(appImageAsset.imageUri),
        containerPort: 8501,
      },
      publicLoadBalancer: true,
      assignPublicIp: true,
      cpu: 512,
      memoryLimitMiB: 1024
    });

This code snippet provisions the VPC, Internet Gateway, Public Subnets, Route Tables, Security Groups, Application Load Balancer, and all ECS Fargate components required to host our application on AWS. We have customized the VPC settings, ECS cluster name, ECS task role and execution role permissions and CloudWatch log group in the stack. We currently run only one ECS task in one Availability Zone for efficient resource utilization. However, this construct can be leveraged to implement automatic scaling when required, ensuring optimal performance as user demand grows.

The docker image is built as part of the CDK assets and will be uploaded to your own AWS account specific CDK assets’ private ECR repository with the below code:

// Build and push Docker image to ECR
    const appImageAsset = new DockerImageAsset(this, 'MyStreamlitAppImage', {
      directory: './lib/docker',
    });

This ECR image is then referenced in the ECS task using ecs.ContainerImage.fromRegistry (appImageAsset.imageUri)

Deploying the Stack

Deploy the stack using the command cdk deployand confirm the deployment by typing “y” and hitting Enter. CDK will create resources based on your code. After completion, it provides a summary. Check the AWS CloudFormation console for stack details to verify the provisioned resources.

Note: Modify CDK code for infrastructure changes and redeploy as needed.

Testing

If the CDK deployment was successful, you will see the Load Balancer DNS name and Fargate Service URL in the outputs section. It will be something like this:

OmrStack.MyFargateServiceServiceURL2AD5256H =http://omr-myfar-1rg2ogk9iabcd-3449675365.us-east-1.elb.amazonaws.com/

Simply open your link in the browser and explore the Music Sheet Transcriber Streamlit App UI as shown below:

Clean-up

To delete all the resources simply run the cdk destroy command. By running this command, you ensure the complete removal of the defined resources, freeing up any allocated resources and eliminating associated costs.

Conclusion

This tutorial serves as a concise guide for seamlessly integrating machine learning into cloud infrastructure for music transcription. You can build your own containerized OMR app on Amazon ECS Fargate at a fraction of cost compared to utilizing paid applications (roughly ~$10) on platforms like the Apple App Store or Google Play Store for this specific use case.

PlainEnglish.io 🚀

Thank you for being a part of the In Plain English community! Before you go:

• Be sure to clap and follow the writer
• Learn how you can also write for In Plain English️
• Follow us: X | LinkedIn | YouTube | Discord | Newsletter
• Visit our other platforms: Stackademic | CoFeed | Venture