Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

Author: torontoai

Running Amazon Elastic Inference Workloads on Amazon ECS

Amazon Elastic Inference (EI) is a new service launched at re:Invent 2018. Elastic Inference reduces the cost of running deep learning inference by up to 75% compared to using standalone GPU instances. Elastic Inference lets you attach accelerators to any Amazon SageMaker or Amazon EC2 instance type and run inference on TensorFlow, Apache MXNet, and ONNX models. Amazon ECS is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to run and scale containerized applications on AWS easily.

In this post, I describe how to accelerate deep learning inference workloads in Amazon ECS by using Elastic Inference. I also demonstrate how multiple containers, running potentially different workloads on the same ECS container instance, can share a single Elastic Inference accelerator. This sharing enables higher accelerator utilization.

As of February 4, 2019, ECS supports pinning GPUs to tasks. This works well for training workloads. However, for inference workloads, using Elastic Inference from ECS is more cost effective when those GPUs are not fully used.

For example, the following diagram shows a cost efficiency comparison of a p2/p3 instance type and a c5.large instance type with each type of Elastic Inference accelerator per 100K single-threaded inference calls (normalized by minimal cost):

TensorFlow: Inference Cost Efficiency with EI

MXNet: Inference Cost Efficiency with EI

Using Elastic Inference on ECS

As an example, this post spins up TensorFlow ModelServer containers as part of an ECS task. You try to identify objects in a single image (the giraffe image that follows), using an SSD with ResNet-50 model, trained with a COCO dataset.

Next, you profile and compare the inference latencies of both a regular and an Elastic Inference–enabled TensorFlow ModelServer. Base your profiling setup on the Elastic Inference with TensorFlow Serving example. You can follow step-by-step instructions or launch an AWS CloudFormation stack with the same infrastructure as this post. Either way, you must be logged into your AWS account as an administrator. For AWS CloudFormation stack creation, choose Launch Stack and follow the instructions.

If Elastic Inference is not supported in the selected Availability Zone, delete and re-create the stack with a different zone. To launch the stack in a Region other than us-east-1, use the same template and template URL. Make sure to select the appropriate Region and Availability Zone.

After choosing Launch Stack, you can also examine the AWS CloudFormation template in detail in AWS CloudFormation Designer.

The AWS CloudFormation stack includes the following resources:

  • A VPC
  • A subnet
  • An Internet gateway
  • An Elastic Inference endpoint
  • IAM roles and policies
  • Security groups and rules
  • Two EC2 instances
    • One for running TensorFlow ModelServer containers (this instance has an Elastic Inference accelerator attached and works as an ECS container instance).
    • One for running a simple client application for making inference calls against the first instance.
  • An ECS task definition

After you create the AWS CloudFormation stack:

  • Go directly to the Running an Elastic Inference-enabled TensorFlow ModelServer task section in this post.
  • Skip Making inference calls.
  • Go directly to Verifying the results.

The second instance runs an example application as part of the bootstrap script.

Make sure to delete the stack once it is no longer needed.

Create an ecsInstanceRole to be used by the ECS container instance

In this step, you create an ecsInstanceRole role to be used by the ECS container instance through an associated instance profile.

In the IAM console, check if an ecsInstanceRole role exists. If the role does not exist, create a new role with the managed policy AmazonEC2ContainerServiceforEC2Role attached and name it ecsInstanceRole. Update its trust policy with the following code:

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Setting up an ECS container instance for Elastic Inference

Your goal is to launch an ECS container instance with an Elastic Inference endpoint attached and with the following additional properties:

Launching the stack automates the setup process. To execute the steps manually, follow the instructions to set up an EC2 instance for Elastic Inference. Make the following changes to these procedures, for simplicity.

Because you plan to call Elastic Inference from ECS tasks, define a task role with relevant permissions. In the IAM console, create a new role with the following properties:

  • Trusted entity type: AWS service
  • Service to use this role: Elastic Container Service
  • Select your use case: Elastic Container Service Task
  • Name: ecs-ei-task-role

In the Attach permissions policies step, select the policy that you created in Set up an EC2 instance for Elastic Inference step. The policy’s content should look like the following example:

{
    "Statement": [
        {
            "Effect": "Allow",
            "Resource": "*",
            "Action": [
                "elastic-inference:Connect",
                "iam:List*",
                "iam:Get*",
                "ec2:Describe*",
                "ec2:Get*"
            ]
        }
    ],
    "Version": "2012-10-17"
}

Only the elastic-inference:Connect permission is required. The remaining permissions provide troubleshooting assistance. You can remove them for production setup.

To validate the role’s trust relationship, on the Trust Relationships tab, choose Show policy document. The policy should look like the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Creating an ECS task execution IAM role

Running on an ECS container instance, the ECS agent needs permissions to make ECS API calls on behalf of the task (for example, pulling container images from ECR). As a result, you must create an IAM role that captures the exact permissions needed. If you’ve created any ECS tasks before, you probably have created this or an equivalent role. For more information, see ECS Task Execution IAM Role.

If no such role exists, in the IAM console, choose Roles and create a new role with the following properties:

  • Trusted entity type: AWS service
  • Service to this role: Elastic Container Service
  • Select your use case: Elastic Container Service Task
  • Name: ecsTaskExecutionRole
  • Attached managed policy: AmazonECSTaskExecutionRolePolicy

Creating a task definition for both regular and Elastic Inference–enabled TensorFlow ModelServer containers

In this step, you create an ECS task definition comprising two containers:

  • One running TensorFlow ModelServer
  • One running an Elastic Inference-enabled TensorFlow ModelServer

Both containers use tensorflow-inference: 1.13-cpu-py27-ubuntu16.04 image (one of the newly released Deep Learning Containers Images). These images already have a regular TensorFlow ModelServer and all its library dependencies. Both containers retrieve and set up the relevant model.

Second container, downloads the Elastic Inference-enabled TensorFlow ModelServer binary. It also removes the ECS_CONTAINER_METADATA_URI environment variable setting to enable Elastic Inference endpoint metadata lookup from the ECS container instance’s metadata:

# install unzip
apt-get --assume-yes install unzip
# download and unzip the model
wget https://s3-us-west-2.amazonaws.com/aws-tf-serving-ei-example/ssd_resnet.zip -P /models/ssdresnet/
unzip -j /models/ssdresnet/ssd_resnet.zip -d /models/ssdresnet/1
# download and extract Elastic Inference enabled TensorFlow Serving
wget https://s3.amazonaws.com/amazonei-tensorflow/tensorflow-serving/v1.13/ubuntu/latest/tensorflow-serving-1-13-1-ubuntu-ei-1-1.tar.gz
tar xzvf tensorflow-serving-1-13-1-ubuntu-ei-1-1.tar.gz
# make the binary executable
chmod +x tensorflow-serving-1-13-1-ubuntu-ei-1-1/amazonei_tensorflow_model_server
# Unset the ECS_CONTAINER_METADATA_URI environment variable to force Elastic Inference endpoint metadata lookup from ECS container instance's metadata.
# Otherwise, Elastic Inference endpoint metadata would tried to be retrieved from container metadata, which would fail.   
env -u ECS_CONTAINER_METADATA_URI tensorflow-serving-1-13-1-ubuntu-ei-1-1/amazonei_tensorflow_model_server --port=${GRPC_PORT} --rest_api_port=${REST_PORT} --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME}

For a regular production setup, I recommend creating a new image from the deep learning container image by turning relevant steps into Dockerfile RUN commands. For this post, you can skip that for simplicity’s sake.

First container downloads model and then, runs the unchanged /usr/bin/tf_serving_entrypoint.sh:

#!/bin/bash 

tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"

In the ECS console, under Task Definitions, choose Create New Task Definitions.

In the Select launch type compatibility dialog box, choose EC2.

In the Create new revision of Task Definition dialog box, scroll to the bottom of the page and choose Configure via JSON.

Paste the following definition into the space provided. Before saving, make sure to replace the two occurrences of <replace-with-your-account-id> with your AWS account ID.

{
    "executionRoleArn": "arn:aws:iam::<replace-with-your-account-id>:role/ecsTaskExecutionRole",
    "containerDefinitions": [
        {
            "entryPoint": [
                "bash",
                "-c",
                "apt-get --assume-yes install unzip; wget https://s3-us-west-2.amazonaws.com/aws-tf-serving-ei-example/ssd_resnet.zip -P ${MODEL_BASE_PATH}/${MODEL_NAME}/; unzip -j ${MODEL_BASE_PATH}/${MODEL_NAME}/ssd_resnet.zip -d ${MODEL_BASE_PATH}/${MODEL_NAME}/1/; /usr/bin/tf_serving_entrypoint.sh"
            ],
            "portMappings": [
                {
                    "hostPort": 8500,
                    "protocol": "tcp",
                    "containerPort": 8500
                },
                {
                    "hostPort": 8501,
                    "protocol": "tcp",
                    "containerPort": 8501
                }
            ],
            "cpu": 0,
            "environment": [
                {
                    "name": "KMP_SETTINGS",
                    "value": "0"
                },
                {
                    "name": "TENSORFLOW_INTRA_OP_PARALLELISM",
                    "value": "2"
                },
                {
                    "name": "MODEL_NAME",
                    "value": "ssdresnet"
                },
                {
                    "name": "KMP_AFFINITY",
                    "value": "granularity=fine,compact,1,0"
                },
                {
                    "name": "MODEL_BASE_PATH",
                    "value": "/models"
                },
                {
                    "name": "KMP_BLOCKTIME",
                    "value": "0"
                },
                {
                    "name": "TENSORFLOW_INTER_OP_PARALLELISM",
                    "value": "2"
                },
                {
                    "name": "OMP_NUM_THREADS",
                    "value": "1"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "image": "763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:1.13-cpu-py27-ubuntu16.04",
            "essential": true,
            "name": "ubuntu-tfs"
        },
        {
            "entryPoint": [
                "bash",
                "-c",
                "apt-get --assume-yes install unzip; wget https://s3-us-west-2.amazonaws.com/aws-tf-serving-ei-example/ssd_resnet.zip -P /models/ssdresnet/; unzip -j /models/ssdresnet/ssd_resnet.zip -d /models/ssdresnet/1; wget https://s3.amazonaws.com/amazonei-tensorflow/tensorflow-serving/v1.13/ubuntu/latest/tensorflow-serving-1-13-1-ubuntu-ei-1-1.tar.gz; tar xzvf tensorflow-serving-1-13-1-ubuntu-ei-1-1.tar.gz; chmod +x tensorflow-serving-1-13-1-ubuntu-ei-1-1/amazonei_tensorflow_model_server; env -u ECS_CONTAINER_METADATA_URI tensorflow-serving-1-13-1-ubuntu-ei-1-1/amazonei_tensorflow_model_server --port=${GRPC_PORT} --rest_api_port=${REST_PORT} --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME}"
            ],
            "portMappings": [
                {
                    "hostPort": 9000,
                    "protocol": "tcp",
                    "containerPort": 9000
                },
                {
                    "hostPort": 9001,
                    "protocol": "tcp",
                    "containerPort": 9001
                }
            ],
            "cpu": 0,
            "environment": [
                {
                    "name": "GRPC_PORT",
                    "value": "9000"
                },
                {
                    "name": "REST_PORT",
                    "value": "9001"
                },
                {
                    "name": "MODEL_NAME",
                    "value": "ssdresnet"
                },
                {
                    "name": "MODEL_BASE_PATH",
                    "value": "/models"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "image": "763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:1.13-cpu-py27-ubuntu16.04",
            "essential": true,
            "name": "ubuntu-tfs-ei"
        }
    ],
    "memory": "2048",
    "taskRoleArn": "arn:aws:iam::<replace-with-your-account-id>:role/ecs-ei-task-role",
    "family": "ei-ecs-ubuntu-tfs-bridge-s3",
    "requiresCompatibilities": [
        "EC2"
    ],
    "networkMode": "bridge",
    "volumes": [],
    "placementConstraints": []
}

You could create an ECS service out of this task definition, but for the sake of this post, you need only run the task.

Running an Elastic Inference–enabled TensorFlow ModelServer task

Make sure to run the task defined in the previous section on the previously created ECS container instance. Register this instance to your default cluster.

In the ECS console, choose Clusters.

Confirm that your EC2 container instance appears in the ECS Instances tab.

Choose TasksRun new task.

For Launch type, select EC2, then pick previously created task (task created by CloudFormation template is named ei-ecs-blog-ubuntu-tfs-bridge) and choose Run Task.

Making inference calls

In this step, you create and run a simple client application to make multiple inference calls using the previously built infrastructure. You also launch an EC2 instance with Deep Learning AMI (DLAMI) on which to run the client application. The TensorFlow library that you use in this example requires the AVX2 instructions set.

Pick the c5.large instance type. Any of the latest generation x86-based EC2 instance types with sufficient memory are fine. The DLAMI provides preinstalled libraries on which TensorFlow relies. Also, because DLAMI is an HVM virtualization type AMI, you can take advantage of the AVX2 instruction set provided by c5.large.

Download labels and an example image to do the inference on:

curl -O https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-paper.txt
curl -O https://s3.amazonaws.com/amazonei/media/3giraffes.jpg

Create a local file named ssd_resnet_client.py, with the following content:

from __future__ import print_function
import grpc
import tensorflow as tf
from PIL import Image
import numpy as np
import time
import os
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc

tf.app.flags.DEFINE_string('server', 'localhost:8500',
                           'PredictionService host:port')
tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format') 
FLAGS = tf.app.flags.FLAGS

if(FLAGS.image == ''):
  print("Supply an Image using '--image [path/to/image]'")
  exit(1)

local_coco_classes_txt = "coco-labels-paper.txt"
 
# Setting default number of predictions
NUM_PREDICTIONS = 20

# Reading coco labels to a list 
with open(local_coco_classes_txt) as f:
  classes = ["No Class"] + [line.strip() for line in f.readlines()]

def main(_):
 
  channel = grpc.insecure_channel(FLAGS.server)
  stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
 
  with Image.open(FLAGS.image) as f:
    f.load()
    
    # Reading the test image given by the user 
    data = np.asarray(f)

    # Setting batch size to 1
    data = np.expand_dims(data, axis=0)

    # Creating a prediction request 
    request = predict_pb2.PredictRequest()
 
    # Setting the model spec name
    request.model_spec.name = 'ssdresnet'
 
    # Setting up the inputs and tensors from image data
    request.inputs['inputs'].CopyFrom(
        tf.contrib.util.make_tensor_proto(data, shape=data.shape))
 
    # Iterating over the predictions. The first inference request can take several seconds to complete

    durations = []

    for curpred in range(NUM_PREDICTIONS): 
      if(curpred == 0):
        print("The first inference request loads the model into the accelerator and can take several seconds to complete. Please standby!")

      # Start the timer 
      start = time.time()
 
      # This is where the inference actually happens 
      result = stub.Predict(request, 60.0)  # 60 secs timeout
      duration = time.time() - start
      durations.append(duration)
      print("Inference %d took %f seconds" % (curpred, duration))

    # Extracting results from output 
    outputs = result.outputs
    detection_classes = outputs["detection_classes"]

    # Creating an ndarray from the output TensorProto
    detection_classes = tf.make_ndarray(detection_classes)

    # Creating an ndarray from the detection_scores
    detection_scores = tf.make_ndarray(outputs['detection_scores'])
 
    # Getting the number of objects detected in the input image from the output of the predictor 
    num_detections = int(tf.make_ndarray(outputs["num_detections"])[0])
    print("%d detection[s]" % (num_detections))

    # Getting the class ids from the output and mapping the class ids to class names from the coco labels with associated detection score
    class_label_score = ["%s: %.2f" % (classes[int(detection_classes[0][index])], detection_scores[0][index]) 
                   for index in range(num_detections)]
    print("SSD Prediction is (label, probability): ", class_label_score)
    print("Latency:")
    for percentile in [95, 50]:
      print("p%d: %.2f seconds" % (percentile, np.percentile(durations, percentile, interpolation='lower')))
 
if __name__ == '__main__':
  tf.app.run()

Make sure to edit the ECS container instance’s security group to permit TCP traffic over ports 8500–8501 and 9000–9001 from the client instance IP address.

From the client instance, check connectivity and the status of the model:

SERVER_IP=<replace-with-ECS-container-instance-IP-address>
for PORT in 8501 9001
do
  curl -s http://${SERVER_IP}:${PORT}/v1/models/ssdresnet
done

Wait until you get two responses like the following:

{
 "model_version_status": [
  {
   "version": "1",
   "state": "AVAILABLE",
   "status": {
    "error_code": "OK",
    "error_message": ""
   }
  }
 ]
}

Then, proceed to run the client application:

source activate amazonei_tensorflow_p27
for PORT in 8500 9000
do
  python ssd_resnet_client.py --server=${SERVER_IP}:${PORT} --image 3giraffes.jpg
done

Verifying the results

The output should be similar to the following:

The first inference request loads the model into the accelerator and can take several seconds to complete. Please standby!
Inference 0 took 12.923095 seconds
Inference 1 took 1.363095 seconds
Inference 2 took 1.338855 seconds
Inference 3 took 1.311022 seconds
Inference 4 took 1.305457 seconds
Inference 5 took 1.303680 seconds
Inference 6 took 1.297357 seconds
Inference 7 took 1.302721 seconds
Inference 8 took 1.299495 seconds
Inference 9 took 1.293291 seconds
Inference 10 took 1.305852 seconds
Inference 11 took 1.292999 seconds
Inference 12 took 1.300874 seconds
Inference 13 took 1.300001 seconds
Inference 14 took 1.297276 seconds
Inference 15 took 1.297859 seconds
Inference 16 took 1.305029 seconds
Inference 17 took 1.315366 seconds
Inference 18 took 1.288984 seconds
Inference 19 took 1.289530 seconds
4 detection[s]
SSD Prediction is (label, probability):  ['giraffe: 0.84', 'giraffe: 0.74', 'giraffe: 0.68', 'giraffe: 0.50']
Latency:
p95: 1.36 seconds
p50: 1.30 seconds
The first inference request loads the model into the accelerator and can take several seconds to complete. Please standby!
Inference 0 took 14.081767 seconds
Inference 1 took 0.295794 seconds
Inference 2 took 0.293941 seconds
Inference 3 took 0.311396 seconds
Inference 4 took 0.291605 seconds
Inference 5 took 0.285228 seconds
Inference 6 took 0.226951 seconds
Inference 7 took 0.283834 seconds
Inference 8 took 0.290349 seconds
Inference 9 took 0.228826 seconds
Inference 10 took 0.284496 seconds
Inference 11 took 0.293179 seconds
Inference 12 took 0.296765 seconds
Inference 13 took 0.230531 seconds
Inference 14 took 0.283406 seconds
Inference 15 took 0.292458 seconds
Inference 16 took 0.300849 seconds
Inference 17 took 0.294651 seconds
Inference 18 took 0.293372 seconds
Inference 19 took 0.225444 seconds
4 detection[s]
SSD Prediction is (label, probability):  ['giraffe: 0.84', 'giraffe: 0.74', 'giraffe: 0.68', 'giraffe: 0.50']
Latency:
p95: 0.31 seconds
p50: 0.29 seconds

If you launched the AWS CloudFormation stack, connect to the client instance with SSH and check the last several lines of this output in /var/log/cloud-init-output.log.

You see a 78% reduction in latency when using an Elastic Inference accelerator with this model and input.

You can launch more than one task and more than one container on the same ECS container instance. You can use the awsvpc network mode if tasks expose the same port numbers. For bridge mode, tasks should expose unique ports.

In multi-task/container scenarios, keep in mind that all clients share accelerator memory. AWS publishes accelerator memory utilization metrics to Amazon CloudWatch as AcceleratorMemoryUsage under the AWS/ElasticInference namespace.

Also, Elastic Inference–enabled containers using the same accelerator must all use either TensorFlow or the MXNet framework. To switch between frameworks, stop and start the ECS container instance.

Conclusion

The described setup shows how multiple deep learning inference workloads running in ECS can be efficiently accelerated by use of Elastic Inference. If inference workload tasks don’t use the entire GPU instance, then using Elastic Inference accelerators may offer an attractive alternative, at a fraction of the cost of dedicated GPU instances. A single accelerator’s capacity can be shared across multiple containers running on the same EC2 container instance, allowing for even greater use of the attached accelerator.


About the Author

Vladimir Mitrovic is a Software Engineer with AWS AI Deep Learning. He is passionate about building fault-tolerant, distributed deep-learning systems. In his spare time, he enjoys solving Project Euler problems.

 

 

 

 

[D] What I’d like to write in my NeurIPS rebuttal

We thank the reviewers for the detailed comments, of which some were even based on our paper.

To the reviewer that said our paper was “underdeveloped” because we didn’t use a different methodology Y from field Z, we’d like to point out that a) this is in field A, b) we provided a framework for how to extend this to other methodologies in field A, and c) methodology Y has no obvious way to extend to the problem we’re addressing (and doing so would be a whole paper in its own right). Do you often read papers and get frustrated that they aren’t the papers you’ve written?

To the same reviewer, who asked why we didn’t cite papers Z1 and Z2, we would again point out that this isn’t field Z and those papers have no relevance to the topic at hand except that you’d have written a paper on a different topic, which we didn’t.

To the reviewer that asked why we didn’t cite X, we’d like to point out that we did cite X, and had a whole paragraph discussing the relationship of this work to that one.

To the reviewer that proposed an example dataset to evaluate our model on, we point out that we already evaluate the model on that data set; see our Experiments section.

To the reviewer that pointed out that our method won’t work when assumption 3 isn’t met, yes, you’re correct. That’s why we stated it as an assumption. Congratulations on your reading comprehension.

To the reviewer that directly copy/pasted our introduction into the “what 3 things does this paper contribute” box, we’ll be sure to include in future revisions a copy/paste-able review justifying “score 10, confidence 5” to make your review easier. That you also confused our main claim with a work we were citing, and otherwise completely missed the discussion on relationship to prior work or what makes this paper novel, makes your review particularly useful to development of the work.

To the reviewer that wrote that, while THEY were familiar with the definitions in a reference, we should explain it for readers that might be confused, we understand entirely. We’ll gladly explain it for “a friend of yours”, err “readers”, and not you, because you get it and you’re smart and it’s just the readers who don’t.

To the reviewer who commented that our results were “contradictory” because we said that our modification “in general performed slightly worse” on this metric, when in fact our plots show it sometimes performed better, we’ll gladly fix our claim to be clear that “in general” doesn’t mean “always” and also our results are even better than the previous wording indicated.

To the reviewer that said our comparison method’s results were worse than reported in the original paper, we’ve carefully compared their bar charts to ours and found that the results are the same to the precision of the graphical printout in the previous paper. If you could lend us your image sharpening function so we can get more significant digits out of their plot, we’d be glad to redo the comparison.

To the reviewer who used half of their review to argue that our entire subfield is dumb and wrong, we thank them for reaching across academic lines to provide commentary in an area that pains you deeply.

And finally, to the reviewers who called our paper (all actual quotes) “original, well-motivated, and worthy of study”, “important in its own right”, that said you “greatly enjoyed reading this paper” and that “this is an interesting problem and certainly worth studying” and that “this paper identifies an important problem … [and the authors] then present a simple” solution, thank you for also marking this a reject. Since all of you gave us scores between 5 and 3, neither the AC nor any of you will ever have to read this response or reconsider your scores before we are inevitably rejected, but we hope that your original, well-motivated, worth-studying, important, interesting, clear papers receive reviews of equal quality in the future!

/salt

submitted by /u/SoFarFromHome
[link] [comments]

[P] Small, fast and simple Python CLI image converter for CNNs.

[P] Small, fast and simple Python CLI image converter for CNNs.

Hello, people) I’ve been working on CLI tool that’s gonna help with converting and dataset augmentation for CNNs or GANs or even other thing that needs images as input data. Here it is https://github.com/liashchynskyi/rudi

https://i.redd.it/labpkay2mgd31.png

Support the repo with a star, if you like it! Thanks, hope this tool will help you))

submitted by /u/liashchynskyi
[link] [comments]

[D] XGBoost doesnt follow the others models

[D] XGBoost doesnt follow the others models

Hello everyone,

I trained my data by different techniques: dimensional analysis (DA), support vector machine (SVM), multi-layer perceptron neural network (ANN) and XGBoost (XGB).The lowest RMSE achieved in the testset was by XGB. However when I tried some combinations of input data the curve was different from other models.

For example:

https://i.redd.it/ol6azkkrggd31.png

This phenomenon is theoretically expressed as a power of type y(x) = a * x ^ b. What are the possible causes for the model curve predicted by XGB not to follow the other models?

Assumptions:

(i) unbalanced continuous data of x or even y (target)? Histogram in https://i.imgur.com/EGg8eNN.png

(ii) hyperparameters (a good mapping of max_depth, min_child_weight, gamma, eta and adding regularizer parameters was tested)

(iii) nature of decision tree conditions

Is there a way I can better generalize (as a post-prune) my model to fit it?

Many thanks!

submitted by /u/drainbamagex
[link] [comments]

A Pigment of Your Imagination: Over Half-Million Images Created with GauGAN AI Art Tool

From amateur doodlers to leading digital artists, creators are coming out in droves to produce masterpieces with NVIDIA’s most popular research demo: GauGAN.

The AI painting web app — which turns rough sketches into stunning, photorealistic scenes — was built to demonstrate NVIDIA Research based on harnessing generative adversarial networks.

More than 500,000 images have been created with GauGAN since the beta version was made publicly available just over a month ago on the NVIDIA AI Playground.

Art directors and concept artists from top film studios and video game companies are among the creative professionals already harnessing GauGAN as a tool to prototype ideas and make rapid changes to synthetic scenes.

“GauGAN popped on the scene and interrupted my notion of what I might be able to use to inspire me,” said Colie Wertz, a concept artist and modeler whose credits include Star Wars, Transformers and Avengers movies. “It’s not something I ever imagined having at my disposal.”

Wertz, using a GauGAN landscape as a foundation, recently created an otherworldly ship design shared on social media.

Colie Wertz ship design
AI Work of Art: Senior concept artist Colie Wertz created this ship design with a GauGAN landscape as a foundation.

“Real-time updates to my environments with a few brush strokes is mind-bending. It’s like instant mood,” said Wertz, who uses NVIDIA RTX GPUs for his creative work. “This is forcing me to reinvestigate how I approach a concept design.”

Attendees of this week’s SIGGRAPH conference can experience GauGAN for themselves in the NVIDIA booth, where it’s running on an HP RTX workstation powered by NVIDIA Quadro RTX GPUs that feature Tensor Cores. NVIDIA researchers will also present GauGAN during a live event at the prestigious computer graphics show.

Users can share their GauGAN creations on Twitter with #SIGGRAPH2019, #GauGAN and @NVIDIADesign to enter our AI art contest, judged by Wertz. Winner will receive an NVIDIA Quadro RTX 6000 GPU.

Unleash Your AI Artist

GauGAN, named for post-Impressionist painter Paul Gauguin, creates photorealistic images from segmentation maps, which are labeled sketches that depict the layout of a scene.

People can use paintbrush and paint bucket tools to design their own landscapes with labels including river, grass, rock and cloud. A style transfer algorithm allows creators to apply filters, modifying the color composition of a generated image, or turning it from a photorealistic scene to a painting.

“As researchers working on image synthesis, we’re always pursuing new techniques to create images with higher fidelity and higher resolution,” said NVIDIA researcher Ming-Yu Liu. “That was our original goal for the project.”

But when the demo was introduced at our GPU Technology Conference in Silicon Valley, it took on a life of its own. Attendees flocked to a tablet on the show floor where they could try it out for themselves, creating stunning scenes of everything from sun-drenched ocean landscapes to idyllic mountain ranges shrouded by clouds.

The latest iteration of the app, on display at SIGGRAPH, lets users upload their own filters to layer onto their masterpieces — adopting the lighting of a perfect sunset photo or emulating the style of a favorite painter.

They can even upload their own landscape images. The AI will convert source images into a segmentation map, which can then be used as a foundation for the user’s artwork.

“We want to make an impact with our research,” Liu said. “This work creates a channel for people to express their creativity and create works of art they wouldn’t be able to do without AI. It’s enabling them to make their imagination come true.”

While the researchers anticipated game developers, landscape designers and urban planners to benefit from this technology, interest in GauGAN has been far more widespread — including from a healthcare organization exploring its use as a therapeutic, stress-mitigating tool for patients.

AI That Captures the Imagination 

Developed using the PyTorch deep learning framework, the neural network behind GauGAN was trained on a million images using the NVIDIA DGX-1 deep learning system. The demo shown at GTC ran on an NVIDIA TITAN RTX GPU, while the web app is hosted on NVIDIA GPUs through Amazon Web Services.

Liu developed the deep neural network and accompanying app along with researchers Taesung Park, Ting-Chun Wang and Jun-Yan Zhu.

The team has publicly released source code for the neural network behind GauGAN, making it available for non-commercial use by other developers to experiment with and build their own applications.

GauGAN is available on the NVIDIA AI Playground for visitors to experience the demo firsthand.

The post A Pigment of Your Imagination: Over Half-Million Images Created with GauGAN AI Art Tool appeared first on The Official NVIDIA Blog.

A Pigment of Your Imagination: GauGAN AI Art Tool Receives “Best of Show,” “Audience Choice” Awards at SIGGRAPH

NVIDIA’s viral real-time AI art application, GauGAN, Tuesday won two major SIGGRAPH awards.

From amateur doodlers to leading digital artists, creators are coming out in droves to produce masterpieces with NVIDIA’s most popular research demo: GauGAN.

And the demo has been a smash hit at the SIGGRAPH professional graphics conference as well, winning both the “Best of Show” and “Audience Choice,” awards at the conference’s Real Time Live competition after NVIDIA’s Ming-Yu Liu, Chris Hebert, Gavriil Klimov and UC Berkeley researcher Taesung Park presented the application to enthusiastic applause.

The AI painting web app — which turns rough sketches into stunning, photorealistic scenes — was built to demonstrate NVIDIA Research based on harnessing generative adversarial networks.

More than 500,000 images have been created with GauGAN since the beta version was made publicly available just over a month ago on the NVIDIA AI Playground.

Art directors and concept artists from top film studios and video game companies are among the creative professionals already harnessing GauGAN as a tool to prototype ideas and make rapid changes to synthetic scenes.

“GauGAN popped on the scene and interrupted my notion of what I might be able to use to inspire me,” said Colie Wertz, a concept artist and modeler whose credits include Star Wars, Transformers and Avengers movies. “It’s not something I ever imagined having at my disposal.”

Wertz, using a GauGAN landscape as a foundation, recently created an otherworldly ship design shared on social media.

Colie Wertz ship design
AI Work of Art: Senior concept artist Colie Wertz created this ship design with a GauGAN landscape as a foundation.

“Real-time updates to my environments with a few brush strokes is mind-bending. It’s like instant mood,” said Wertz, who uses NVIDIA RTX GPUs for his creative work. “This is forcing me to reinvestigate how I approach a concept design.”

Attendees of this week’s SIGGRAPH conference can experience GauGAN for themselves in the NVIDIA booth, where it’s running on an HP RTX workstation powered by NVIDIA Quadro RTX GPUs that feature Tensor Cores. NVIDIA researchers will also present GauGAN during a live event at the prestigious computer graphics show.

Users can share their GauGAN creations on Twitter with #SIGGRAPH2019, #GauGAN and @NVIDIADesign to enter our AI art contest, judged by Wertz. Winner will receive an NVIDIA Quadro RTX 6000 GPU.

Unleash Your AI Artist

GauGAN, named for post-Impressionist painter Paul Gauguin, creates photorealistic images from segmentation maps, which are labeled sketches that depict the layout of a scene.

People can use paintbrush and paint bucket tools to design their own landscapes with labels including river, grass, rock and cloud. A style transfer algorithm allows creators to apply filters, modifying the color composition of a generated image, or turning it from a photorealistic scene to a painting.

“As researchers working on image synthesis, we’re always pursuing new techniques to create images with higher fidelity and higher resolution,” said NVIDIA researcher Ming-Yu Liu. “That was our original goal for the project.”

But when the demo was introduced at our GPU Technology Conference in Silicon Valley, it took on a life of its own. Attendees flocked to a tablet on the show floor where they could try it out for themselves, creating stunning scenes of everything from sun-drenched ocean landscapes to idyllic mountain ranges shrouded by clouds.

The latest iteration of the app, on display at SIGGRAPH, lets users upload their own filters to layer onto their masterpieces — adopting the lighting of a perfect sunset photo or emulating the style of a favorite painter.

They can even upload their own landscape images. The AI will convert source images into a segmentation map, which can then be used as a foundation for the user’s artwork.

“We want to make an impact with our research,” Liu said. “This work creates a channel for people to express their creativity and create works of art they wouldn’t be able to do without AI. It’s enabling them to make their imagination come true.”

While the researchers anticipated game developers, landscape designers and urban planners to benefit from this technology, interest in GauGAN has been far more widespread — including from a healthcare organization exploring its use as a therapeutic, stress-mitigating tool for patients.

AI That Captures the Imagination 

Developed using the PyTorch deep learning framework, the neural network behind GauGAN was trained on a million images using the NVIDIA DGX-1 deep learning system. The demo shown at GTC ran on an NVIDIA TITAN RTX GPU, while the web app is hosted on NVIDIA GPUs through Amazon Web Services.

Liu developed the deep neural network and accompanying app along with researchers Taesung Park, Ting-Chun Wang and Jun-Yan Zhu.

The team has publicly released source code for the neural network behind GauGAN, making it available for non-commercial use by other developers to experiment with and build their own applications.

GauGAN is available on the NVIDIA AI Playground for visitors to experience the demo firsthand.

Note: This post has been updated from the original to reflect the results  of Tuesday’s Real Time Live competition at SIGGRAPH. 

The post A Pigment of Your Imagination: GauGAN AI Art Tool Receives “Best of Show,” “Audience Choice” Awards at SIGGRAPH appeared first on The Official NVIDIA Blog.

SIGGRAPH Showcases Amazing NVIDIA Research Breakthroughs, NVIDIA Wins Best in Show Award

Get ready to dig in this week.

SIGGRAPH is here and we’re helping graphics professionals, researchers, developers and students of all kinds take advantage of the latest advances in graphics, including new possibilities in real-time ray tracing, AI, and augmented reality.

SIGGRAPH is the most important computer graphics conference in the world, and our research team and collaborators from top universities and many industries are here with us.

At the top of the list: ray tracing, using NVIDIA’s RTX platform, which fuses ray tracing, deep learning and rasterization. We’re directly involved in 34 of 50 ray tracing-related technical sessions this week — far more than any other company. And our talks are drawing luminaries from around the industry, with four technical Academy Award winners participating in NVIDIA sponsored sessions.

Beyond the technical sessions, we’ll be showcasing new developer tools, and giving attendees a first-hand look at some of our most exciting work. One great example is NVIDIA GauGAN an interactive paint program that uses GANs (generative adversarial networks) to create works of art from simple brush strokes. Now everybody can be an artist.

Never been to the moon? A stunning new demo virtually transports visitors to the Apollo 11 landing site using never-before-shown AI pose estimation that captures their body movements in real time. This all became possible by combining NVIDIA Omniverse technology, AI and RTX ray tracing.

The story behind all these stories: our 200-person strong NVIDIA Research team — spread across 11 worldwide locations. The group embodies NVIDIA’s commitment to bringing innovative new ideas to customers in everything from machine learning, computer vision, self-driving cars, robotics, graphics, computer architecture, programming systems and more.

A Host of Papers, Talks, Tutorials

We’ll be leading or participating in six SIGGRAPH courses that detail various facets of the next-generation graphics technologies we’ve played a leading role in bringing to market.

These courses touch on everything from an introduction to real-time ray tracing, the use of the NVIDIA OptiX API, Monte Carlo and quasi-Monte Carlo sampling techniques, the latest in path tracing techniques, open problems in real-time rendering, and the future of ray tracing as a whole.

The common denominator: RTX. The real-time ray-tracing capabilities RTX unleashes offer far more realistic lighting effects than traditional real-time rendering techniques.

We’re also sponsoring seven courses on topics ranging from deep learning for content creation and real-time rendering to GPU ray tracing for film and design.

And we’re presenting technical papers that detail how our latest near-eye AR display demo works and take the next leap in denoising Monte Carlo rendering using convolutional neural networks — a cornerstone of AI — effectively using modern AI techniques to greatly reduce the time required to generate realistic images.

The Eyes Have It: Prescription-Embedded AR Display Wins Best in Show Award

You’ll be able to get hands-on with our latest technology in SIGGRAPH’s Emerging Technologies area. That’s where we have a pair of wearable augmented reality displays technology you need to see, especially if you don’t see very well without regular eyeglasses.

The first, “Prescription AR,” is a prescription-embedded AR display that won a SIGGRAPH Best in Show Emerging Technology award Monday.

The display is many times thinner and lighter and has a wider field of view that current-generation AR devices. Virtual objects appear throughout the natural instead of clustered in the center, and it has your prescription built right into it if you wear corrective optics. This much closer to the goal of comfortable, practical and socially-acceptable AR displays than anything currently available.

 

The second research demonstration, “Foveated AR,” is a headset that adapts to your gaze in real time using deep learning. It adjusts the resolution of the images it displays and their focal depth to match wherever you are looking and gives both sharper images and a wider field of view than any previous AR display.

To do this, it combines two different displays per eye, a high-resolution small field of view displaying images to the portion of the human retina where visual acuity is highest, and a low-resolution display for peripheral vision. The result is high-quality visual experiences with reduced power and computation.

TITAN RTX Giveway

Finally, NVIDIA is thanking the student volunteer community at SIGGRAPH with a daily giveaway of TITAN RTX while exhibit hall is open. These students are the future of one of the world’s most vibrant professional communities, a community we’re privileged to be a part of.

The post SIGGRAPH Showcases Amazing NVIDIA Research Breakthroughs, NVIDIA Wins Best in Show Award appeared first on The Official NVIDIA Blog.