Deep learning is changing the world. However, much of the foundation work, such as building containers, can slow you down. This post describes how you can build, train, and deploy fastai models into Amazon SageMaker training and hosting by using the Amazon SageMaker Python SDK and a PyTorch base image. This helps you avoid the extra steps of building your own container.

Amazon SageMaker is a fully managed machine learning (ML) service. It allows data scientists and developers to quickly build, train, and deploy ML models into production at a low cost. The Amazon SageMaker Python SDK is an open source library for training and hosting ML models. It is easy to use and compatible with popular deep learning frameworks such as TensorFlow, MXNet, PyTorch, and Chainer. AWS recently added the fastai library to the base PyTorch container. This allows you to take advantage of the fastai deep learning model in Amazon SageMaker, instead of providing your own container.

Using modern best practices, the fastai library helps create advanced deep learning models with just a few lines of code. This includes domains like computer vision, natural language processing, structured data, or collaborative filtering.

The organization fast.ai develops and maintains the fastai library, which works with the popular deep learning open source PyTorch package. The organization recently placed well in the DAWNBench Competition. It also provides popular online courses to train developers in using their models, even those with no background or experience in ML.

Set up the environment

To set up a new Amazon SageMaker notebook instance with the fastai library installed, choose Launch Stack:

This AWS CloudFormation template provisions all the AWS resources that you need for this walkthrough. To create the stack, select I acknowledge that AWS CloudFormation might create IAM resources and choose Create stack.

When the stack has been successfully completed, AWS CloudFormation announces the stack’s CREATE_COMPLETE state.

In the Amazon SageMaker console, choose Notebook instances.

You can see a notebook instance labeled fastai, created from the AWS CloudFormation stack. Select Open Juypter to launch a managed Juypter environment. You can write your own Python code here to build the model.

For this post, we provide the IPython notebook under SageMaker Examples, Advanced Functionality, fastai_lesson1_sagemaker_example.ipynb. Create a copy of this notebook to build, train, and deploy the model into Amazon SageMaker.

For demonstration purpose, we use the Oxford IIIT Pet dataset to identify dog breeds.

Because the fastai library builds onto PyTorch, you can use the Amazon SageMaker Python SDK to train your PyTorch model. Training a PyTorch model is a two-step process:

1. Create a PyTorch training script.
2. Create a training job in Amazon SageMaker.

Create a PyTorch training script

Define a _train() function where you write your training code. A typical training script loads data from the input channels, configures training with hyperparameters, trains a model, and saves a model to model_dir to host later. For a more detailed example, see the pets.py file in the fastai_oxford_pets GitHub repo.

Here is the basic training code:

print('Creating DataBunch object')
    data = ImageDataBunch.from_name_re(path_img, fnames, pat, 
                                ds_tfms=get_transforms(), 
                                size=args.image_size, 
                                bs=args.batch_size).normalize(imagenet_stats)

    # create the CNN model
    print('Create CNN model from model zoo')
    print(f'Model architecture is {args.model_arch}')
    arch = getattr(models, args.model_arch)
    print("Creating pretrained conv net")    
    learn = create_cnn(data, arch, metrics=error_rate)
    print('Fit for 4 cycles')
    learn.fit_one_cycle(4)    
    learn.unfreeze()
    print('Unfreeze and fit for another 2 cycles')
    learn.fit_one_cycle(2, max_lr=slice(1e-6,1e-4))
    print('Finished Training')

In this example, combine the training and inference code in a single script. Make sure to put your training code in a main guard (if __name__==’__main__’:), so that Amazon SageMaker doesn’t inadvertently run it at the wrong point in execution. You can also place pass hyperparameters in your training script, which you can retrieve with an argparse.ArgumentParser. The following code snippet features passing hyperparameters:

if __name__ == '__main__':
    parser = argparse.ArgumentParser()

    parser.add_argument('--workers', type=int, default=2, metavar='W',
                        help='number of data loading workers (default: 2)')
    parser.add_argument('--epochs', type=int, default=2, metavar='E',
                        help='number of total epochs to run (default: 2)')
    parser.add_argument('--batch_size', type=int, default=64, metavar='BS',
                        help='batch size (default: 4)')
    parser.add_argument('--lr', type=float, default=0.001, metavar='LR',
                        help='initial learning rate (default: 0.001)')

Create a training job in Amazon SageMaker

You can train and host PyTorch models on Amazon SageMaker with the help of the PyTorch base image and PyTorch SageMaker Estimators, which handle end-to-end training and hosting.

The entry_point argument takes the training script that contains the _train() function. The train_instance_type argument takes the instance type where the model trains. If you set this parameter as ‘local’, the model trains on Amazon SageMaker notebook instance as a Docker container.

To use local mode training, install Docker Compose, as well as NVIDIA-Docker if training with a GPU. To train the model on Amazon SageMaker, pass the value of a GPU-optimized instance type, such as ml.p3.2xlarge or ml.p2.xlarge.

Amazon SageMaker bases the instance lifetime on your training time. When it completes training, Amazon SageMaker uploads the model artifact to Amazon S3 and terminates the instance.

The training script contains the definition of the model and the serving functions that are required to reconstruct the model back at the endpoint. This doesn’t limit when the code was written or the model was created, as long as the models and code match versioning.

pets_estimator = PyTorch(entry_point='source/pets.py',
                         base_job_name='fastai-pets',
                         role=role,
                         framework_version='1.0.0',
                         train_instance_count=1,
                         train_instance_type='ml.p3.2xlarge')

pets_estimator.fit(data_location)

Deploy the model in Amazon SageMaker

Using the Amazon SageMaker Python SDK, create a PyTorchModel object from the PyTorch estimator. The deploy () method on the model object creates an Amazon SageMaker endpoint, which serves prediction requests in real time.

The Amazon SageMaker endpoint runs an Amazon SageMaker-provided PyTorch model server. It hosts the model that your training script produces after you call fit. This was the model you saved to model_dir.

pets_model=PyTorchModel(model_data=pets_estimator.model_data,
                        name=pets_estimator._current_job_name,
                        role=role,
                        framework_version=pets_estimator.framework_version,
                        entry_point=pets_estimator.entry_point,
                        predictor_cls=ImagePredictor)

pets_predictor = pets_model.deploy(initial_instance_count=1, 
                                     instance_type='ml.c5.large')

Create the inference script

To do the inference, define four functions in your inference script. In this example, we use the same script for training and inference. You define these functions along with a training function called _train():

• model_fn(model_dir)—Loads the model into the Amazon SageMaker PyTorch model server.
• input_fn(request_body,request_content_type)—Deserializes the data into an object for predictions.
• predict_fn(input_object,model)—On deserialized data performs inference against loaded model.
• output_fn(prediction,content_type)—Serializes predictions according to the response content type.

For a more detailed example, see the pets.py file in the fastai_oxford_pets GitHub repo.

Run your inference using predict () method after the model deploys. This method returns the result of inference against your model.

response = pets_predictor.predict(img_bytes)

Clean up

Make sure that you stop the notebook instance and delete the Amazon SageMaker endpoint to prevent any additional charges.

Conclusion

In this post, we showed you how to use Amazon SageMaker to train and host fastai models using the PyTorch base image and Amazon SageMaker Python SDK. This approach eliminates the bring-your-own-container approach, which helps you to build, train, and deploy a fastai model easily and quickly in Amazon SageMaker.

To get started, login to the AWS Management Console and deploy the AWS CloudFormation template. If you are a new user, you will need to create an account. Also, follow this GitHub link for detail information on how to build the fastai model with Amazon SageMaker.

About the authors:

Amit Mukherjee is a partner solutions architect with AWS. He provides architectural guidance to help partners achieve success in the cloud. He has interest in the deep learning space. In his spare time, he enjoys spending quality time with his family.

Matt McClean is a partner solutions architect for AWS. He is a specialist in machine learning and works with technology partners in the EMEA Region, providing guidance on developing solutions using AWS technologies. In his spare time, he is a passionate skier and cyclist.

Customers often ask us how to get started when they do not have a deep data science and machine learning (ML) background. At AWS, our goal is to put ML in the hands of every developer and data scientist.

AWS Training and Certification has partnered with edX to help you get started quickly and easily with ML with our interactive course, Amazon SageMaker: Simplifying Machine Learning Application Development. 

Available exclusively on edX, Amazon SageMaker: Simplifying Machine Learning Application Development is an intermediate-level digital course that provides a baseline understanding of ML and how applications can be built, trained, and deployed using Amazon SageMaker. Amazon SageMaker is a fully managed, modular service that covers the entire ML workflow. It helps you label and prepare your data; choose an algorithm; train the model; tune and optimize it for deployment; make predictions; and act.

This course was developed by AWS experts. It explains the following:

• Key problems that ML can address and ultimately help solve
• How to train a model using Amazon SageMaker’s built-in algorithms and a Jupyter Notebook instance
• How to deploy a model using Amazon SageMaker
• How to integrate the published SageMaker endpoint with an application

Amazon SageMaker: Simplifying Machine Learning Application Development is recommended for developers of all skills. If you want to take your models from concept to production quickly and easily, or are looking to level up in ML, this course is for you. Before beginning, we recommend that you have at least one year of software development experience. You should also have a basic understanding of AWS services and the AWS Management Console, either through previous experience or the AWS Developer Professional Series.

The course is divided into four weekly lessons, with an estimated two to four hours per week of study time. It features video-based lectures, demonstrations, and hands-on lab exercises. You can set your own pace and deadlines. Everyone can take the weekly quizzes, which are not graded and allow unlimited retries.

This on-demand, 100%-digital course is available now and is offered on a complimentary basis.

Get started today at Amazon SageMaker: Simplifying Machine Learning Application Development.

About the Author

Jennifer Davis is the Senior Manager, Product Marketing, for AWS Training and Certification.

When Electronic Caregiver’s founder and CEO, Anthony Dohrmann, started the company a decade ago, he was reacting to a difficult situation faced by 100 million Americans and countless individuals globally: the challenge of managing health treatment for chronic diseases. “Patients are often confused about their care instructions and non-adherence with care plans and medications schedules are estimated to cause 50% of all treatment failures,” he explains.

As such, Electronic Caregiver was designed “to improve the patient experience and to positively engage patients in their personal care plans. We improve communication between providers, families, and caregivers, and expedite a more informed response to the need of the aging and ill. We intend to reduce costly complications, improve health outcomes, and extend lifespans.”

Today, Electronic Caregiver’s solution revolves around Addison, a-state-of-the-art, 3D-animated virtual caregiver. She can engage in two-way conversations and is programmed for a user’s personal needs. Similar to a human in-home caregiver, Addison monitors patients’ activity, reminds them to take medications, collects vitals, and conducts real-time health assessments—all from the safety and comfort of a patient’s home. Whereas a patient would otherwise need to make myriad doctor visits or pay an in-home caregiver, Addison brings health solutions to the user wherever they are.

To power that life-changing magic, Electronic Caregiver relies on AWS in multiple ways. For raw computational power to store patient data in a HIPPA-compliant way, the team uses services including AWS Lambda functions. For the patient-facing experience, Electronic Caregiver has developed an augmented reality (AR) character named Addison using Amazon Sumerian. And for the intelligence behind that character including collecting and analyzing data, AWS IoT Core, AWS IoT Greengrass, and Amazon SageMaker are key to the solution. The architecture that the Electronic Caregiver team has developed is shown in the following diagram.

Bryan Chasko, CTO at Electronic Caregiver, comments, “We saw an opportunity to use the latest sensing, artificial intelligence, and other cloud-based technologies to address unmet customer needs with a fuller-featured solution than traditional alert devices.”

Specifically, Electronic Caregiver provides patients with wearable gadgets (such as a wrist pendant) and monitoring devices (such as a contact-free thermometer and a glucose meter) that are connected to the cloud.

To connect device fleets, AWS IoT Core easily and securely connects devices to the cloud via an MQTT lightweight communication protocol specifically designed to tolerate intermittent connections, minimize the code footprint on devices, and reduce network bandwidth requirements.

Whenever a user completes a health reading, such as checking their temperature or completing a physical therapy exercise, that activity generates data that the devices capture. That data can then be queried to check whether a measured value is in the expected range. If the reading is good, the patient will receive a contextually appropriate positive response from Addison.

For example, in cases where a patient is in physical therapy recovering from an injury, Electronic Caregiver monitors improvement to their range of motion. The built-in gamification rewards the patient with points and even sends gifts to their homes to celebrate improvements. This personalized support reinforces their ongoing commitment to their treatment plan and helps these patients execute their treatments properly; it is pivotal to their long-term recovery.

If a reading is atypical, Electronic Caregiver springs into action to get the patient back on track. From a technical perspective, AWS IoT Greengrass Machine Learning Inference pushes a machine learning model built in Amazon SageMaker directly to the edge device in the user’s home. The patient is asked specific questions to help assess the cause of the anomaly, and then they receive from their device a prediction of the likely reason(s) for this result as well as recommended solutions. These questions and solutions are voiced to the patient with Amazon Lex and Amazon Polly, as well as shared with the patient’s selected stakeholders (such as family members and doctors) so everyone on the individual’s care team is immediately aware.

With this set-up, it is as though the patient has a constant caregiver watching out for them, so they receive the quality of care typical of a full-time facility like a nursing home, but possible from their own house. As a further benefit, even individuals who are not co-located with the patient (such as family members on a different continent) can get real-time updates from across the world.

In addition to the connected wearables, Electronic Caregiver has developed a platform to track patients’ activity and ensure that they are conscious and mobile. If activity is not detected, Electronic Caregiver can summon emergency response, coming to the rescue quickly in the event of a fall or other lapse into unconsciousness.

There is a visual analytics monitoring system that also enables personalized monitored medication reminders. Motion is tracked by the visual analytics system and then the pills are identified with Amazon SageMaker-trained machine learning models. This means that Addison can pinpoint when a user has taken their medication and remind them if they’re late.

Amazon Lex also accepts verbal input from the user, so a patient can simply articulate that they’re taking a medication and the system logs it. This feature makes it feel almost like the caregiver is human. Just as someone would articulate to a housemate that they’d completed their medication routine, they can inform Addison.

“Only 3% of the US population can afford live caregiving,” Dohrmann notes. “We are bringing affordable, effective care alternatives to the world through Addison.”

About the Author

Marisa Messina is on the AWS ML marketing team, where her job includes identifying the most innovative AWS-using customers and showcasing their inspiring stories. Prior to AWS, she worked on consumer-facing hardware and then university-facing cloud offerings at Microsoft. Outside of work, she enjoys exploring the Pacific Northwest hiking trails, cooking without recipes, and dancing in the rain.

Amazon SageMaker Ground Truth helps you build highly accurate training datasets for machine learning. It offers easy access to public and private human labelers, and provides them with built-in workflows and interfaces for common labeling tasks. Ground Truth can lower your labeling costs by up to 70% using automatic labeling. It works by training Ground Truth from human-labeled data, so that the service learns to label data independently.

In addition to built-in workflows, Ground Truth gives you the option to upload custom workflows. A custom workflow consists of an HTML interface that provides the human labelers with all of the instructions and required tools for completing the labeling tasks. You also create pre– and post-processing AWS Lambda functions:

• The pre-processing Lambda function helps customize input to the HTML interface.
• The post-processing Lambda function helps to process the data. For example, one of its primary uses is to host an accuracy improvement algorithm to tell Ground Truth how it should assess the quality of human-provided labels.

An algorithm is used to find consensus on what is “right” when the same data is provided to multiple human labelers. It also identifies and de-emphasizes those labelers who tend to provide poor quality data. You can upload the HTML interface and the pre- and post-processing Lambda functions using the Amazon SageMaker console.

To integrate successfully with the HTML interface, the pre– and post-processing Lambda functions should adhere to the input/output specifications laid out in the Creating Custom Labeling Workflows. Setting up all the moving pieces and getting them to talk to each other successfully may take a few iterations.

In this post, I walk you through the process of setting up a custom workflow with a custom HTML template and sample Lambda functions. The sample Lambda functions can be found in the AWS Serverless Application Repository. These Lambda functions can be easily deployed to your AWS account and directly modified in the AWS Lambda Console. The source code is available in the aws-sagemaker-ground-truth-recipe GitHub repo.

For this post, you create a custom labeling job for instance segmentation. But first, deploy Lambda functions from the AWS Serverless Application Repository to your AWS account.

Import Lambda functions

On the Serverless Application Repository home page, select “Available applications” on the left-hand menu and search for Ground Truth. Choose aws-sagemaker-ground-truth-recipe.

On the application’s details page, choose Deploy. Make sure that the user has permissions to create IAM roles. If the user does not have permissions, this deployment fails.

It may take a few minutes to deploy this application. Wait until you see the status screen, which shows that four AWS resources (two Lambda functions and two IAM roles) have been created.

Now, you have successfully imported the Lambda functions used in the labeling job into your account. To modify these Lambda functions, select them and tweak the Python code.

Create a custom labeling job

Assume that there are millions of images taken from cameras mounted in cars driving the public roadways. These images are stored in an Amazon S3 bucket location called s3://mybucket/datasets/streetscenes/. To start a labeling job for instance segmentation, you first create a manifest to be fed to Ground Truth.

The following code example shows the sample contents of a manifest file with a set of images. For more information, see Input Data.

{"source-ref": "S3 location for image 1"} 
{"source-ref": "S3 location for image 2"} 
   ... 
{"source-ref": "S3 location for image n"} 

Step 1: Download the example dataset

If you already have a manifest file for instance segmentation, skip this section.

For this example, I use the CBCL StreetScenes dataset. This dataset has over 3000 images, but I use a selection of just 10 images. The full dataset is approximately 2 GB. You can choose to upload all of the images to S3 for labeling or just a selection of them.

• Download the zip file and extract to a folder. By default, the folder is Output.
• Create a small sample dataset with which to work:

  mkdir streetscenes
  cp Original/SSDB00010.JPG ./streetscenes/
  cp Original/SSDB00017.JPG ./streetscenes/
  cp Original/SSDB00019.JPG ./streetscenes/
  cp Original/SSDB00025.JPG ./streetscenes/
  cp Original/SSDB00038.JPG ./streetscenes/
  cp Original/SSDB00016.JPG ./streetscenes/
  cp Original/SSDB00018.JPG ./streetscenes/
  cp Original/SSDB00021.JPG ./streetscenes/
  cp Original/SSDB00029.JPG ./streetscenes/
  cp Original/SSDB00039.JPG ./streetscenes/

In the S3 console, create the /streetscenes folder in your bucket.  S3 is a key-value store, so there is no concept of folders. However, the S3 console gives you a sense of folder structure by using forward slashes in the key. You use the console to create the key.

Upload the following files to your S3 bucket, s3://mybucket/datasets/streetscenes/. You can use the S3 console or the following AWS CLI command:

aws s3 sync streetscenes/ s3://gt-recipe-demo/datasets/streetscenes/
upload: streetscenes/SSDB00010.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00010.JPG
upload: streetscenes/SSDB00017.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00017.JPG
upload: streetscenes/SSDB00018.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00018.JPG
upload: streetscenes/SSDB00021.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00021.JPG
upload: streetscenes/SSDB00025.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00025.JPG
upload: streetscenes/SSDB00038.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00038.JPG
upload: streetscenes/SSDB00029.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00029.JPG
upload: streetscenes/SSDB00016.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00016.JPG
upload: streetscenes/SSDB00039.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00039.JPG
upload: streetscenes/SSDB00019.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00019.JPG

Step 2: Create an input manifest

If you already have a manifest file for instance segmentation, skip this section.

In the Amazon SageMaker console, start the process by creating a labeling job.

Under input dataset location, choose Create manifest file. This tool helps you create the manifest by crawling an S3 location containing raw data (images or text).

For images, the crawler takes an input s3Prefix and crawls all of the image files with extensions .jpg, .jpeg, and .png in that prefix. It then creates a manifest with each line as follows:

{"source-ref":"<s3-location-of-crawled-image>"}

The Create manifest file link opens a modal window. Enter the S3 path to which you uploaded the images files, and make sure to include the trailing slash. Next, choose Create. When the creation process is completed, choose Use this manifest. It takes a few seconds to create the manifest.

In this example, the objects are images in S3, so you can use the crawling tool to create the initial manifest. Each line of JSON contains a field called source-ref pointing to the s3Uri value of an image. The contents of the created manifest file should look as follows:

{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00010.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00016.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00017.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00018.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00019.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00021.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00025.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00029.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00038.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00039.JPG"}

Step 3: Create a custom labeling job

Configure the following job settings:

• Labeling job name—This name must be unique in your account within an AWS Region.
• Input dataset S3 location—The location of the input manifest file that you created in Step 2.
• Output dataset S3 location—The location to which output data is written.
• IAM role —Sets permissions using an IAM policy. Make a note of this role, as you need it in Step 4.

Select the custom task type and choose Next.

For Workers, choose Private. For more information about the different workforce options, see Managing Your Workforce.

There are a number of labeling UI templates that you can use for setting up your own custom workflows. In this case, use the instance segmentation UI. For Templates, choose Instance Segmentation.

Modify the HTML code to look like the following. In the original template, you had three placeholders: src, header, and labels. I changed the header and labels fields. When tasks are created for workers using this template, Ground Truth provides the data to fill in the src placeholder field.

<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>

<crowd-form>
  <crowd-instance-segmentation
    name="annotatedResult"
    src="{{ task.input.taskObject | grant_read_access }}"
    header="Draw a polygon around all people in the image."
    labels="['Person']"
  >
    <full-instructions header="Segmentation Instructions">
      <ol>
          <li><strong>Read</strong> the task carefully and inspect the image.</li>
          <li><strong>Read</strong> the options and review the examples provided to understand more about the labels.</li>
          <li><strong>Choose</strong> the appropriate label that best suits the image.</li>
      </ol>
    </full-instructions>

    <short-instructions>
      <p>Use the tools to label all instances of the requested items in the image</p>
    </short-instructions>
  </crowd-instance-segmentation>
</crowd-form>

Next, for pre– and post-processing task Lambda function fields, select the Lambda functions that you imported earlier.

Under Custom labeling task setup, choose Preview. Remember to allow pop-ups before attempting to preview the UI. If the page loads successfully without errors, you know that the pre-processing task Lambda function and the custom HTML template are working well together.

Step 4: Give execute permissions to the Amazon SageMaker role

In the previous step, while creating a Ground Truth labeling job, you created an IAM role. Ground Truth uses this IAM role to execute your labeling job. This role should trust the execution role of the post-processing Lambda function.

In the Lambda console, select the Lambda function that you previously imported. At the top of the page or under the Tags section, note the Amazon Resource Name (ARN). It should look like the following:

arn:aws:lambda:us-east-1:919226420625:function:serverlessrepo-aws-sagema-GtRecipeAnnotationConsol-xxxxxxx

Choose Execution role, Use an existing role, and view the role.

Copy the IAM role ARN.

In the IAM console, find the Amazon SageMaker execution role that you created. Choose Trust relationships, Edit trust relationship.  Add the copied Lambda execution role to the trust relationship. The following code example shows the contents of the trust relationship.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::<your-aws-account>:role/serverlessrepo-aws-sagema-GtRecipeAnnotationConsol-xxxxx"
        ],
        "Service": [
          "lambda.amazonaws.com",
          "sagemaker.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Choose Permissions, Attach policies. Select AWSLambdaFullAccess, and choose Attach Policy. After attaching the policy, your Permissions tab should look like the following screenshot:

Close the current tab.

Step 5: Submit the labeling job

In the main browser tab, which has your labeling job open, choose Submit. The labeling job is in progress. Wait for this job to complete.

If you have workers assigned to your private work team, instruct them to work on your tasks. If you have added yourself as a worker, complete the tasks in the private work team portal. For more information, see Managing a Private Workforce.

Labeling results

After the workers perform the labeling work, your output manifest looks like the following:

{"source-ref":"s3://gt-recipe-demo/dataset/streetscenes/SSDB00010.JPG","gt-label":{"annotationsFromAllWorkers":[{"workerId":"public.us-east-1.M52TVGWFFYHS34QM3EHF3NMTKY","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"}},{"workerId":"public.us-east-1.JZUQXKHROY2DBQ2V2ZLEWEYCAE","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAAZQTFRFAAAALKAsCO/WQQAAAAJ0Uk5TAP9bkSK1AAABeklEQVR4nO3awWnEQAxAURsffHQJLsWlOaVtKVNCjjmEzGYbGMNqQTK8V8FHJyE0TQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC8ac8OuLL/ZBdcOPojO2FMYNQpMKh8YBcYJDBo7r1lNwyVD1xuEPid3TD0H1h731pvEPib3TC09f6X3TBUPnDvvWc3DAmMOqoHlp/gK/ArO2JEYJTAqO0OgY/siBGBUQKj1jsEtuyIEYFRAqNegaWPMwKjbhHYsiNGBEYJjBIYJTBqqR44Vw+cBEad1QOP6oF79cBNYNBaPbD800L5t49JYNRZPfCoHrhVD5yrB05nyy64sLXsggtLyy640rIDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPicJ/PI71EA2TUjAAAAAElFTkSuQmCC"}}}"}},{"workerId":"public.us-east-1.H7WTTAHGRYWYXEA7V7E7KOMOCE","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"}}]},"gt-label-17-metadata":{"type":"groundtruth/custom","job-name":"gt-label-17","human-annotated":"yes","creation-date":"2019-04-01T05:21:34+0000"}}
{"source-ref":"s3://gt-recipe-demo/dataset/streetscenes/SSDB00016.JPG","gt-label":{"annotationsFromAllWorkers":[{"workerId":"public.us-east-1.H7WTTAHGRYWYXEA7V7E7KOMOCE","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"},{"color":"#1f77b4","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAgMAAAAXsjzqAAAAAXNSR0IB2cksfwAAAAlQTFRFAAAALKAsH3e00sbfdAAAAAN0Uk5TAP//RFDWIQAABI1JREFUeJzt3E2O4kgUhVGwhNTyqCdswqtgCTUo74elMGx5lV1JklWGWXP10u2IcxYQL/QJ/2CEDwcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAaMhpnufL1pvYMwFDHwF/bL2JPRMw9BHw59ab2DMBQx8B5603sWcChu4BL1vvYscEDN0Duo95n4Che8CG72PG6gGtB5yuxQPuAdu9jxmWW/GExgOOyz/FEz4DXoqnbGZaluIJjQdcluVaO+EzYKv3McOvgLfaEZ8BWz0J/jqCq0+Cj4CX2ilbWT7UjngEbPNOcLgHvJbOeARs8yM43gPeSmd8BWzyMjLdA9aeBL8CtngZ+TyCi0+C57ndj+D4CHgtnHH86tfiZWR6BLwVzvh9BLd4GfmOgLOAkWPTAZf6gCcBM7OAkWPTAYf6gOc+AtZ9FZkFjBzbDjiWBzwLmJkFjJwaDzhVBzwLmOkmYNUDwdYDLgJmBAwJmBkEzNQHnAXMNB5wFDCzCnitmSBgqPGAk4AZAUMChpbqgEcBMx0FvJUMaDzgIGBGwJCAoVHATH3Ak4AZAUMdBaz5YbjxgIOAIQFDAoYmATPlAc+NBxwFzKzvY0oGtB5w8AkMVX8T6SjgtWT95gNOtUdwRwFvNes3H3CsPYJffxZuL+Dvy3DR+t0ErPqXQ/MBD7WnwH4CXouWbz/gVHoK7CbgN/1Xs8WAY+kR/PqzcIMBh9IjuJeA3/R//yYDHipvYroJeK1avIeAU+EpsIuAY+U7YzoJeCtbvIeAQ+XbK19/Fm4zYOELaHsIePiuVwe2G/Bat3YXAafCtbsI+Ffh2l0ErCRgSMDQWcCMgCEBQwKGBAwJGBIw9NpPwP/q9SN42XpDu9P6a5DrNf4q+HoChgQMCRgSMCRgSMCQgCEBQwKGBAwJGBIwJGBIwJCAIQFDAoYEDAkYEjAkYOg54Na72SEBQwKGBAwJGBIw9BTw59a72SEBQwKGBAydBMw8Bfyx9W52SMCQgCEBQwKGngJett7NDgkYEjAkYEjAkIChp4Bbb2aPBAwJGBIwJGBoHdDz1DcIGBIwJGBIwNA6oOepbxAwJGBIwJCAIQFD64CXrTezRwKGBAwJGBIwtA649V526SxgRsCQgCEBQ6uAnqe+Q8CQgCEBQwKGBAytAnqe+g4BQwKGBAwJGBIwtAp42XovuyRgSMCQgCEBQ6uAW29lnwQMCRgSMCRgSMDQn4AeSL9FwJCAIQFDAob+BPQ89S0ChmYBMwKGBIyd9Usd9Yv9vfUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAID/t38Buw1HOPtmWvcAAAAASUVORK5CYIIu003d"}}}"}},{"workerId":"public.us-east-1.M52TVGWFFYHS34QM3EHF3NMTKY","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"}},{"workerId":"public.us-east-1.QUERNWRJ6HNNO5D6XDUVPMWTMA","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"},{"color":"#1f77b4","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAgMAAAAXsjzqAAAAAXNSR0IB2cksfwAAAAlQTFRFAAAALKAsH3e00sbfdAAAAAN0Uk5TAP//RFDWIQAABJZJREFUeJzt3EFuIjsUhtFUJKQW89oEq2AJb/BqPyyFYYtVdpMOCcms+9etUl2fswDb+oRlMAUvLwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEB3y3nrFezbImBiWgRM3PsJmBAwJGDoHvC/rRexZwKGBAwJGBIwJGBIwNA94P9bL2LPBAwJGBIwNAuYETAkYOgecNl6EXvWPuDrtXb89gGP19rx2we8/awdv3vA4+1WO0H3gDcBI6+/A15KZ5h7fy13+h3wWjpD64D31191wKVxwONbv+JjWMDMJGDmIGCmdcDTCgHfDuGuX8utEXAZIGDpR5E/AZt+FFkh4PQe8Fw4x3ZWCHh4D9jzUv+2XsCee3iFgPMjYMtjZIWAj3499/Aj4KVuio+AHY+R1/qA02fAhnt4hYCHz4ANj5GVA57LZtnKCgHnp4D9jpEVAi5L55fg8RHwWjbFl4DtjpFTecCp9yuwPuChd8CbgJHX+oCzgJmldcCPQ7juTr93wFN5wEnAzKF3wJuAkdf6gLOAmaV1wM9DuOxOv3fAU/krcBIw8+0M6XYlfRMwUx9w7h2wfgsLGGoe8Chgpv6NdPOA9Z+Fxwl4rZmge8CTgJmjgBkBQ+VfKnUPWP5oR/uAL+sG7Pd01scmvtYM3z/gYxNfakYfIOCLgKE/m7ho8BECvm3ilb7VFPBvDRPwWjT2MkrAS9HYAoaGCVg19igBV3q8UsC/NkrAa9XY3wK2+53Nm2PhD70EDI0SsGzsQQLW/W+RgKFBAl7Lxh4k4KVsbAFDgwSsG1vA0BgBC/99UcDQGAGvdWN/C3ium2lDhQGb/1Dp3fFSNrSAoUEC1g0tYGiMgD/qhhYwNEbAQgKGBAwJGBIwJGDoe8Ct17M7AoYEDAkYEjAkYEjAkICh5n+dVU/AkC0cEjAlYGj+0q/nz0RKCRg6CJiZBMwImBIwNAuYETD05Rju+YRvrUnAjIAhAUMChgQMCRgSMCRg6EvA89ar2SEBUwKGBAwJGBIwJGBIwJCAoeeAW69llwQMCRgSMCRgSMCQgCEBQ0/9PNnxLwQMCRgSMCRgSMCQgCEBQ08BPZjwLwQMCRgSMCRg6Cngeeu17JKAIQFDAoaeAm69lH0SMCRgSMCQy5iQgCEBQwKGBAy5jAkJGBIwJGDIZUxIwJCAIQFDLmNCAoYEDAkYEjDkMiYkYEjAkIAhlzEhAUMChgQMCRhymxUSMCRgSMCQgCEBQwKGBAwJGBIwJGBIwNBHwK0XslcChgQMCRg6CBiaBcxMAoYOAoYEDB0EDM0CAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAjOIXhsjo6mjj6oAAAAAASUVORK5CYIIu003d"}}}"}}]},"gt-label-17-metadata":{"type":"groundtruth/custom","job-name":"gt-label-17","human-annotated":"yes","creation-date":"2019-04-01T05:21:34+0000"}}

Expand one JSON line to view the annotations. You can see that three workers worked on the image and produced annotations:

• source-ref: The location of the image.
• workerId: The ID of the worker to whom the subsequent annotationData In this case, you can see three workerIds, which means three workers annotated this image.
• annotationData: The annotation result.
• gt-label-17-metadata: The metadata associated with the labeling job of which this image was a part.

{  
   "source-ref":"s3://gt-recipe-demo/dataset/streetscenes/SSDB00010.JPG",
   "gt-label-17":{  
      "annotationsFromAllWorkers":[  
         {  
            "workerId":"public.us-east-1.M52TVGWFFYHS34QM3EHF3NMTKY",
            "annotationData":{  
               "content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"
            }
         },
         {  
            "workerId":"public.us-east-1.JZUQXKHROY2DBQ2V2ZLEWEYCAE",
            "annotationData":{  
               "content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAAZQTFRFAAAALKAsCO/WQQAAAAJ0Uk5TAP9bkSK1AAABeklEQVR4nO3awWnEQAxAURsffHQJLsWlOaVtKVNCjjmEzGYbGMNqQTK8V8FHJyE0TQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC8ac8OuLL/ZBdcOPojO2FMYNQpMKh8YBcYJDBo7r1lNwyVD1xuEPid3TD0H1h731pvEPib3TC09f6X3TBUPnDvvWc3DAmMOqoHlp/gK/ArO2JEYJTAqO0OgY/siBGBUQKj1jsEtuyIEYFRAqNegaWPMwKjbhHYsiNGBEYJjBIYJTBqqR44Vw+cBEad1QOP6oF79cBNYNBaPbD800L5t49JYNRZPfCoHrhVD5yrB05nyy64sLXsggtLyy640rIDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPicJ/PI71EA2TUjAAAAAElFTkSuQmCC"}}}"
            }
         },
         {  
            "workerId":"public.us-east-1.H7WTTAHGRYWYXEA7V7E7KOMOCE",
            "annotationData":{  
               "content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"
            }
         }
      ]
   },
   "gt-label-17-metadata":{  
      "type":"groundtruth/custom",
      "job-name":"gt-label",
      "human-annotated":"yes",
      "creation-date":"2019-04-01T05:21:34+0000"
   }
}

Cleanup

In order to avoid incurring future charges:

1. Make sure that your labeling job is marked as “Complete,” “Stopped,” or “Failed” in the Amazon SageMaker console.
2. Delete the corresponding S3 bucket “mybucket” in Amazon S3.
3. Delete the “serverlessrepo-aws-sagemaker-ground-truth-recipe” stack from Amazon CloudFormation console.

Conclusion

In this post, I started by deploying the pre– and post-processing Lambda functions from a Ground Truth app, using the AWS Serverless Application Repository. I then created a custom labeling job and configured it to use the imported Lambda functions.

These sample Lambda functions help you get a custom labeling job running quickly. You can add or modify them with your own logic, using the AWS Lambda Console.

Visit your AWS Management Console to get started!

About the Authors

Anjan Dash is a Software Development Engineer in AWS AI where he builds large scale distributed systems to solve complex machine learning problems. He is primarily focused on innovating technologies that can ‘Divide and Conquer’ Big Data problem. In his spare time, he loves spending time with family in outdoors activities.

Revekka Kostoeva is a Software Developer Engineer intern at Amazon AI where she works on customer facing and internal solutions to expand the breadth of Sagemaker Ground Truth services. As a researcher, she is driven to improve the tools of the trade to drive innovation forward.