Amazon SageMaker Ground Truth helps you build highly accurate training datasets for machine learning. It offers easy access to public and private human labelers, and provides them with built-in workflows and interfaces for common labeling tasks. Ground Truth can lower your labeling costs by up to 70% using automatic labeling. It works by training Ground Truth from human-labeled data, so that the service learns to label data independently.
In addition to built-in workflows, Ground Truth gives you the option to upload custom workflows. A custom workflow consists of an HTML interface that provides the human labelers with all of the instructions and required tools for completing the labeling tasks. You also create pre– and post-processing AWS Lambda functions:
- The pre-processing Lambda function helps customize input to the HTML interface.
- The post-processing Lambda function helps to process the data. For example, one of its primary uses is to host an accuracy improvement algorithm to tell Ground Truth how it should assess the quality of human-provided labels.
An algorithm is used to find consensus on what is “right” when the same data is provided to multiple human labelers. It also identifies and de-emphasizes those labelers who tend to provide poor quality data. You can upload the HTML interface and the pre- and post-processing Lambda functions using the Amazon SageMaker console.
To integrate successfully with the HTML interface, the pre– and post-processing Lambda functions should adhere to the input/output specifications laid out in the Creating Custom Labeling Workflows. Setting up all the moving pieces and getting them to talk to each other successfully may take a few iterations.
In this post, I walk you through the process of setting up a custom workflow with a custom HTML template and sample Lambda functions. The sample Lambda functions can be found in the AWS Serverless Application Repository. These Lambda functions can be easily deployed to your AWS account and directly modified in the AWS Lambda Console. The source code is available in the aws-sagemaker-ground-truth-recipe GitHub repo.
For this post, you create a custom labeling job for instance segmentation. But first, deploy Lambda functions from the AWS Serverless Application Repository to your AWS account.
Import Lambda functions
On the Serverless Application Repository home page, select “Available applications” on the left-hand menu and search for Ground Truth. Choose aws-sagemaker-ground-truth-recipe.
On the application’s details page, choose Deploy. Make sure that the user has permissions to create IAM roles. If the user does not have permissions, this deployment fails.
It may take a few minutes to deploy this application. Wait until you see the status screen, which shows that four AWS resources (two Lambda functions and two IAM roles) have been created.
Now, you have successfully imported the Lambda functions used in the labeling job into your account. To modify these Lambda functions, select them and tweak the Python code.
Create a custom labeling job
Assume that there are millions of images taken from cameras mounted in cars driving the public roadways. These images are stored in an Amazon S3 bucket location called s3://mybucket/datasets/streetscenes/. To start a labeling job for instance segmentation, you first create a manifest to be fed to Ground Truth.
The following code example shows the sample contents of a manifest file with a set of images. For more information, see Input Data.
{"source-ref": "S3 location for image 1"}
{"source-ref": "S3 location for image 2"}
...
{"source-ref": "S3 location for image n"}
Step 1: Download the example dataset
If you already have a manifest file for instance segmentation, skip this section.
For this example, I use the CBCL StreetScenes dataset. This dataset has over 3000 images, but I use a selection of just 10 images. The full dataset is approximately 2 GB. You can choose to upload all of the images to S3 for labeling or just a selection of them.
- Download the zip file and extract to a folder. By default, the folder is Output.
- Create a small sample dataset with which to work:
mkdir streetscenes
cp Original/SSDB00010.JPG ./streetscenes/
cp Original/SSDB00017.JPG ./streetscenes/
cp Original/SSDB00019.JPG ./streetscenes/
cp Original/SSDB00025.JPG ./streetscenes/
cp Original/SSDB00038.JPG ./streetscenes/
cp Original/SSDB00016.JPG ./streetscenes/
cp Original/SSDB00018.JPG ./streetscenes/
cp Original/SSDB00021.JPG ./streetscenes/
cp Original/SSDB00029.JPG ./streetscenes/
cp Original/SSDB00039.JPG ./streetscenes/
In the S3 console, create the /streetscenes folder in your bucket. S3 is a key-value store, so there is no concept of folders. However, the S3 console gives you a sense of folder structure by using forward slashes in the key. You use the console to create the key.
Upload the following files to your S3 bucket, s3://mybucket/datasets/streetscenes/. You can use the S3 console or the following AWS CLI command:
aws s3 sync streetscenes/ s3://gt-recipe-demo/datasets/streetscenes/
upload: streetscenes/SSDB00010.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00010.JPG
upload: streetscenes/SSDB00017.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00017.JPG
upload: streetscenes/SSDB00018.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00018.JPG
upload: streetscenes/SSDB00021.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00021.JPG
upload: streetscenes/SSDB00025.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00025.JPG
upload: streetscenes/SSDB00038.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00038.JPG
upload: streetscenes/SSDB00029.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00029.JPG
upload: streetscenes/SSDB00016.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00016.JPG
upload: streetscenes/SSDB00039.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00039.JPG
upload: streetscenes/SSDB00019.JPG to s3://gt-recipe-demo/datasets/streetscenes/SSDB00019.JPG
Step 2: Create an input manifest
If you already have a manifest file for instance segmentation, skip this section.
In the Amazon SageMaker console, start the process by creating a labeling job.
Under input dataset location, choose Create manifest file. This tool helps you create the manifest by crawling an S3 location containing raw data (images or text).
For images, the crawler takes an input s3Prefix and crawls all of the image files with extensions .jpg, .jpeg, and .png in that prefix. It then creates a manifest with each line as follows:
{"source-ref":"<s3-location-of-crawled-image>"}
The Create manifest file link opens a modal window. Enter the S3 path to which you uploaded the images files, and make sure to include the trailing slash. Next, choose Create. When the creation process is completed, choose Use this manifest. It takes a few seconds to create the manifest.
In this example, the objects are images in S3, so you can use the crawling tool to create the initial manifest. Each line of JSON contains a field called source-ref pointing to the s3Uri value of an image. The contents of the created manifest file should look as follows:
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00010.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00016.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00017.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00018.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00019.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00021.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00025.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00029.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00038.JPG"}
{"source-ref":"s3://gt-recipe-demo/datasets/streetscenes/SSDB00039.JPG"}
Step 3: Create a custom labeling job
Configure the following job settings:
Select the custom task type and choose Next.
For Workers, choose Private. For more information about the different workforce options, see Managing Your Workforce.
There are a number of labeling UI templates that you can use for setting up your own custom workflows. In this case, use the instance segmentation UI. For Templates, choose Instance Segmentation.
Modify the HTML code to look like the following. In the original template, you had three placeholders: src, header, and labels. I changed the header and labels fields. When tasks are created for workers using this template, Ground Truth provides the data to fill in the src placeholder field.
<script src="https://assets.crowd.aws/crowd-html-elements.js"></script>
<crowd-form>
<crowd-instance-segmentation
name="annotatedResult"
src="{{ task.input.taskObject | grant_read_access }}"
header="Draw a polygon around all people in the image."
labels="['Person']"
>
<full-instructions header="Segmentation Instructions">
<ol>
<li><strong>Read</strong> the task carefully and inspect the image.</li>
<li><strong>Read</strong> the options and review the examples provided to understand more about the labels.</li>
<li><strong>Choose</strong> the appropriate label that best suits the image.</li>
</ol>
</full-instructions>
<short-instructions>
<p>Use the tools to label all instances of the requested items in the image</p>
</short-instructions>
</crowd-instance-segmentation>
</crowd-form>
Next, for pre– and post-processing task Lambda function fields, select the Lambda functions that you imported earlier.
Under Custom labeling task setup, choose Preview. Remember to allow pop-ups before attempting to preview the UI. If the page loads successfully without errors, you know that the pre-processing task Lambda function and the custom HTML template are working well together.
Step 4: Give execute permissions to the Amazon SageMaker role
In the previous step, while creating a Ground Truth labeling job, you created an IAM role. Ground Truth uses this IAM role to execute your labeling job. This role should trust the execution role of the post-processing Lambda function.
In the Lambda console, select the Lambda function that you previously imported. At the top of the page or under the Tags section, note the Amazon Resource Name (ARN). It should look like the following:
arn:aws:lambda:us-east-1:919226420625:function:serverlessrepo-aws-sagema-GtRecipeAnnotationConsol-xxxxxxx
Choose Execution role, Use an existing role, and view the role.
Copy the IAM role ARN.
In the IAM console, find the Amazon SageMaker execution role that you created. Choose Trust relationships, Edit trust relationship. Add the copied Lambda execution role to the trust relationship. The following code example shows the contents of the trust relationship.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<your-aws-account>:role/serverlessrepo-aws-sagema-GtRecipeAnnotationConsol-xxxxx"
],
"Service": [
"lambda.amazonaws.com",
"sagemaker.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
Choose Permissions, Attach policies. Select AWSLambdaFullAccess, and choose Attach Policy. After attaching the policy, your Permissions tab should look like the following screenshot:
Close the current tab.
Step 5: Submit the labeling job
In the main browser tab, which has your labeling job open, choose Submit. The labeling job is in progress. Wait for this job to complete.
If you have workers assigned to your private work team, instruct them to work on your tasks. If you have added yourself as a worker, complete the tasks in the private work team portal. For more information, see Managing a Private Workforce.
Labeling results
After the workers perform the labeling work, your output manifest looks like the following:
{"source-ref":"s3://gt-recipe-demo/dataset/streetscenes/SSDB00010.JPG","gt-label":{"annotationsFromAllWorkers":[{"workerId":"public.us-east-1.M52TVGWFFYHS34QM3EHF3NMTKY","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"}},{"workerId":"public.us-east-1.JZUQXKHROY2DBQ2V2ZLEWEYCAE","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAAZQTFRFAAAALKAsCO/WQQAAAAJ0Uk5TAP9bkSK1AAABeklEQVR4nO3awWnEQAxAURsffHQJLsWlOaVtKVNCjjmEzGYbGMNqQTK8V8FHJyE0TQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC8ac8OuLL/ZBdcOPojO2FMYNQpMKh8YBcYJDBo7r1lNwyVD1xuEPid3TD0H1h731pvEPib3TC09f6X3TBUPnDvvWc3DAmMOqoHlp/gK/ArO2JEYJTAqO0OgY/siBGBUQKj1jsEtuyIEYFRAqNegaWPMwKjbhHYsiNGBEYJjBIYJTBqqR44Vw+cBEad1QOP6oF79cBNYNBaPbD800L5t49JYNRZPfCoHrhVD5yrB05nyy64sLXsggtLyy640rIDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPicJ/PI71EA2TUjAAAAAElFTkSuQmCC"}}}"}},{"workerId":"public.us-east-1.H7WTTAHGRYWYXEA7V7E7KOMOCE","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"}}]},"gt-label-17-metadata":{"type":"groundtruth/custom","job-name":"gt-label-17","human-annotated":"yes","creation-date":"2019-04-01T05:21:34+0000"}}
{"source-ref":"s3://gt-recipe-demo/dataset/streetscenes/SSDB00016.JPG","gt-label":{"annotationsFromAllWorkers":[{"workerId":"public.us-east-1.H7WTTAHGRYWYXEA7V7E7KOMOCE","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"},{"color":"#1f77b4","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAgMAAAAXsjzqAAAAAXNSR0IB2cksfwAAAAlQTFRFAAAALKAsH3e00sbfdAAAAAN0Uk5TAP//RFDWIQAABI1JREFUeJzt3E2O4kgUhVGwhNTyqCdswqtgCTUo74elMGx5lV1JklWGWXP10u2IcxYQL/QJ/2CEDwcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAaMhpnufL1pvYMwFDHwF/bL2JPRMw9BHw59ab2DMBQx8B5603sWcChu4BL1vvYscEDN0Duo95n4Che8CG72PG6gGtB5yuxQPuAdu9jxmWW/GExgOOyz/FEz4DXoqnbGZaluIJjQdcluVaO+EzYKv3McOvgLfaEZ8BWz0J/jqCq0+Cj4CX2ilbWT7UjngEbPNOcLgHvJbOeARs8yM43gPeSmd8BWzyMjLdA9aeBL8CtngZ+TyCi0+C57ndj+D4CHgtnHH86tfiZWR6BLwVzvh9BLd4GfmOgLOAkWPTAZf6gCcBM7OAkWPTAYf6gOc+AtZ9FZkFjBzbDjiWBzwLmJkFjJwaDzhVBzwLmOkmYNUDwdYDLgJmBAwJmBkEzNQHnAXMNB5wFDCzCnitmSBgqPGAk4AZAUMChpbqgEcBMx0FvJUMaDzgIGBGwJCAoVHATH3Ak4AZAUMdBaz5YbjxgIOAIQFDAoYmATPlAc+NBxwFzKzvY0oGtB5w8AkMVX8T6SjgtWT95gNOtUdwRwFvNes3H3CsPYJffxZuL+Dvy3DR+t0ErPqXQ/MBD7WnwH4CXouWbz/gVHoK7CbgN/1Xs8WAY+kR/PqzcIMBh9IjuJeA3/R//yYDHipvYroJeK1avIeAU+EpsIuAY+U7YzoJeCtbvIeAQ+XbK19/Fm4zYOELaHsIePiuVwe2G/Bat3YXAafCtbsI+Ffh2l0ErCRgSMDQWcCMgCEBQwKGBAwJGBIw9NpPwP/q9SN42XpDu9P6a5DrNf4q+HoChgQMCRgSMCRgSMCQgCEBQwKGBAwJGBIwJGBIwJCAIQFDAoYEDAkYEjAkYOg54Na72SEBQwKGBAwJGBIw9BTw59a72SEBQwKGBAydBMw8Bfyx9W52SMCQgCEBQwKGngJett7NDgkYEjAkYEjAkIChp4Bbb2aPBAwJGBIwJGBoHdDz1DcIGBIwJGBIwNA6oOepbxAwJGBIwJCAIQFD64CXrTezRwKGBAwJGBIwtA649V526SxgRsCQgCEBQ6uAnqe+Q8CQgCEBQwKGBAytAnqe+g4BQwKGBAwJGBIwtAp42XovuyRgSMCQgCEBQ6uAW29lnwQMCRgSMCRgSMDQn4AeSL9FwJCAIQFDAob+BPQ89S0ChmYBMwKGBIyd9Usd9Yv9vfUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAID/t38Buw1HOPtmWvcAAAAASUVORK5CYIIu003d"}}}"}},{"workerId":"public.us-east-1.M52TVGWFFYHS34QM3EHF3NMTKY","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"}},{"workerId":"public.us-east-1.QUERNWRJ6HNNO5D6XDUVPMWTMA","annotationData":{"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"},{"color":"#1f77b4","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAgMAAAAXsjzqAAAAAXNSR0IB2cksfwAAAAlQTFRFAAAALKAsH3e00sbfdAAAAAN0Uk5TAP//RFDWIQAABJZJREFUeJzt3EFuIjsUhtFUJKQW89oEq2AJb/BqPyyFYYtVdpMOCcms+9etUl2fswDb+oRlMAUvLwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEB3y3nrFezbImBiWgRM3PsJmBAwJGDoHvC/rRexZwKGBAwJGBIwJGBIwNA94P9bL2LPBAwJGBIwNAuYETAkYOgecNl6EXvWPuDrtXb89gGP19rx2we8/awdv3vA4+1WO0H3gDcBI6+/A15KZ5h7fy13+h3wWjpD64D31191wKVxwONbv+JjWMDMJGDmIGCmdcDTCgHfDuGuX8utEXAZIGDpR5E/AZt+FFkh4PQe8Fw4x3ZWCHh4D9jzUv+2XsCee3iFgPMjYMtjZIWAj3499/Aj4KVuio+AHY+R1/qA02fAhnt4hYCHz4ANj5GVA57LZtnKCgHnp4D9jpEVAi5L55fg8RHwWjbFl4DtjpFTecCp9yuwPuChd8CbgJHX+oCzgJmldcCPQ7juTr93wFN5wEnAzKF3wJuAkdf6gLOAmaV1wM9DuOxOv3fAU/krcBIw8+0M6XYlfRMwUx9w7h2wfgsLGGoe8Chgpv6NdPOA9Z+Fxwl4rZmge8CTgJmjgBkBQ+VfKnUPWP5oR/uAL+sG7Pd01scmvtYM3z/gYxNfakYfIOCLgKE/m7ho8BECvm3ilb7VFPBvDRPwWjT2MkrAS9HYAoaGCVg19igBV3q8UsC/NkrAa9XY3wK2+53Nm2PhD70EDI0SsGzsQQLW/W+RgKFBAl7Lxh4k4KVsbAFDgwSsG1vA0BgBC/99UcDQGAGvdWN/C3ium2lDhQGb/1Dp3fFSNrSAoUEC1g0tYGiMgD/qhhYwNEbAQgKGBAwJGBIwJGDoe8Ct17M7AoYEDAkYEjAkYEjAkICh5n+dVU/AkC0cEjAlYGj+0q/nz0RKCRg6CJiZBMwImBIwNAuYETD05Rju+YRvrUnAjIAhAUMChgQMCRgSMCRg6EvA89ar2SEBUwKGBAwJGBIwJGBIwJCAoeeAW69llwQMCRgSMCRgSMCQgCEBQ0/9PNnxLwQMCRgSMCRgSMCQgCEBQ08BPZjwLwQMCRgSMCRg6Cngeeu17JKAIQFDAoaeAm69lH0SMCRgSMCQy5iQgCEBQwKGBAy5jAkJGBIwJGDIZUxIwJCAIQFDLmNCAoYEDAkYEjDkMiYkYEjAkIAhlzEhAUMChgQMCRhymxUSMCRgSMCQgCEBQwKGBAwJGBIwJGBIwNBHwK0XslcChgQMCRg6CBiaBcxMAoYOAoYEDB0EDM0CAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAjOIXhsjo6mjj6oAAAAAASUVORK5CYIIu003d"}}}"}}]},"gt-label-17-metadata":{"type":"groundtruth/custom","job-name":"gt-label-17","human-annotated":"yes","creation-date":"2019-04-01T05:21:34+0000"}}
Expand one JSON line to view the annotations. You can see that three workers worked on the image and produced annotations:
- source-ref: The location of the image.
- workerId: The ID of the worker to whom the subsequent annotationData In this case, you can see three workerIds, which means three workers annotated this image.
- annotationData: The annotation result.
- gt-label-17-metadata: The metadata associated with the labeling job of which this image was a part.
{
"source-ref":"s3://gt-recipe-demo/dataset/streetscenes/SSDB00010.JPG",
"gt-label-17":{
"annotationsFromAllWorkers":[
{
"workerId":"public.us-east-1.M52TVGWFFYHS34QM3EHF3NMTKY",
"annotationData":{
"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"
}
},
{
"workerId":"public.us-east-1.JZUQXKHROY2DBQ2V2ZLEWEYCAE",
"annotationData":{
"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[{"color":"#2ca02c","label":"Person"}],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAAZQTFRFAAAALKAsCO/WQQAAAAJ0Uk5TAP9bkSK1AAABeklEQVR4nO3awWnEQAxAURsffHQJLsWlOaVtKVNCjjmEzGYbGMNqQTK8V8FHJyE0TQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC8ac8OuLL/ZBdcOPojO2FMYNQpMKh8YBcYJDBo7r1lNwyVD1xuEPid3TD0H1h731pvEPib3TC09f6X3TBUPnDvvWc3DAmMOqoHlp/gK/ArO2JEYJTAqO0OgY/siBGBUQKj1jsEtuyIEYFRAqNegaWPMwKjbhHYsiNGBEYJjBIYJTBqqR44Vw+cBEad1QOP6oF79cBNYNBaPbD800L5t49JYNRZPfCoHrhVD5yrB05nyy64sLXsggtLyy640rIDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPicJ/PI71EA2TUjAAAAAElFTkSuQmCC"}}}"
}
},
{
"workerId":"public.us-east-1.H7WTTAHGRYWYXEA7V7E7KOMOCE",
"annotationData":{
"content":"{"annotatedResult":{"inputImageProperties":{"height":960,"width":1280},"instances":[],"labeledImage":{"pngImageData":"iVBORw0KGgoAAAANSUhEUgAABQAAAAPAAQMAAABQEkY6AAAAAXNSR0IB2cksfwAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAACsSURBVHic7cExAQAAAMKg9U/tbwagAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAN1veAAH/xLK9AAAAAElFTkSuQmCC"}}}"
}
}
]
},
"gt-label-17-metadata":{
"type":"groundtruth/custom",
"job-name":"gt-label",
"human-annotated":"yes",
"creation-date":"2019-04-01T05:21:34+0000"
}
}
Cleanup
In order to avoid incurring future charges:
- Make sure that your labeling job is marked as “Complete,” “Stopped,” or “Failed” in the Amazon SageMaker console.
- Delete the corresponding S3 bucket “mybucket” in Amazon S3.
- Delete the “serverlessrepo-aws-sagemaker-ground-truth-recipe” stack from Amazon CloudFormation console.
Conclusion
In this post, I started by deploying the pre– and post-processing Lambda functions from a Ground Truth app, using the AWS Serverless Application Repository. I then created a custom labeling job and configured it to use the imported Lambda functions.
These sample Lambda functions help you get a custom labeling job running quickly. You can add or modify them with your own logic, using the AWS Lambda Console.
Visit your AWS Management Console to get started!
About the Authors
Anjan Dash is a Software Development Engineer in AWS AI where he builds large scale distributed systems to solve complex machine learning problems. He is primarily focused on innovating technologies that can ‘Divide and Conquer’ Big Data problem. In his spare time, he loves spending time with family in outdoors activities.
Revekka Kostoeva is a Software Developer Engineer intern at Amazon AI where she works on customer facing and internal solutions to expand the breadth of Sagemaker Ground Truth services. As a researcher, she is driven to improve the tools of the trade to drive innovation forward.