Keras is a popular and well-documented open source library for deep learning, while Amazon SageMaker provides you with easy tools to train and optimize machine learning models. Until now, you had to build a custom container to use both, but Keras is now part of the built-in TensorFlow environments for TensorFlow and Apache MXNet. Not only does this simplify the development process, it also allows you to use standard Amazon SageMaker features like script mode or automatic model tuning.

Keras’s excellent documentation, numerous examples, and active community make it a great choice for beginners and experienced practitioners alike. The library provides a high-level API that makes it easy to build all kind of deep learning architectures, with the option to use different backends for training and prediction: TensorFlow, Apache MXNet, and Theano.

In this post, I show you how to train and deploy Keras 2.x models on Amazon SageMaker, using the built-in TensorFlow environments for TensorFlow and Apache MXNet. In the process, you also learn the following:

• To run the same Keras code on Amazon SageMaker that you run on your local machine, use script mode.
• To optimize hyperparameters, launch automatic model tuning.
• Deploy your models with Amazon Elastic Inference.

The Keras example

This example demonstrates training a simple convolutional neural network on the Fashion MNIST dataset. This dataset replaces the well-known MNIST dataset. It has the same number of classes (10), samples (60,000 for training, 10,000 for validation), and image properties (28×28 pixels, black and white). But it’s also much harder to learn, which makes for a more interesting challenge.

First, set up TensorFlow as your Keras backend (and switch to Apache MXNet later on). For more information, see the mnist_keras_tf_local.py script.

The process is straightforward:

• Grab optional parameters from the command line, or use default values if they’re missing.
• Download the dataset and save it to the /data directory.
• Normalize the pixel values, and one hot encode labels.
• Build the convolutional neural network.
• Train the model.
• Save the model to TensorFlow Serving format for deployment.

Positioning your image channels can be tricky. Black and white images have a single channel (black), while color images have three channels (red, green, and blue). The library expects data to have a well-defined shape when training a model, describing the batch size, the height and width of images, and the number of channels. TensorFlow specifically requires the input shape formatted as (batch size, width, height, channels), with channels last. Meanwhile, MXNet expects (batch size, channels, width, height), with channels first. To avoid training issues created by using the wrong shape, I add a few lines of code to identify the active setting and reshape the dataset to compensate.

Now check that this code works by running it on a local machine, without using Amazon SageMaker.

$ python mnist_keras_tf_vanilla.py
Using TensorFlow backend.
channels_last
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
<output removed>
Validation loss    : 0.2472819224089384
Validation accuracy: 0.9126

Training and deploying the Keras model

You must make a few minimal changes, but script mode does most of the work for you. Before invoking your code inside the TensorFlow environment, Amazon SageMaker sets four environment variables

• SM_NUM_GPUS—The number of GPUs present on the instance.
• SM_MODEL_DIR— The output location for the model.
• SM_CHANNEL_TRAINING— The location of the training dataset.
• SM_CHANNEL_VALIDATION—The location of the validation dataset.

You can use these values in your training code with just a simple modification:

parser.add_argument('--gpu-count', type=int, default=os.environ['SM_NUM_GPUS'])
parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR'])
parser.add_argument('--training', type=str, default=os.environ['SM_CHANNEL_TRAINING'])
parser.add_argument('--validation', type=str, default=os.environ['SM_CHANNEL_VALIDATION'])

What about hyperparameters? No work needed there. Amazon SageMaker passes them as command line arguments to your code.

For more information, see the updated script, mnist_keras_tf.py.

Training on Amazon SageMaker

After deploying your Keras model, you can begin training on Amazon SageMaker. For more information, see the Fashion MNIST-SageMaker.ipynb notebook.

The process is straightforward:

• Download the dataset.
• Define the training and validation channels.
• Configure the TensorFlow estimator, enabling script mode and passing some hyperparameters.
• Train, deploy, and predict.

In the training log, you can see how Amazon SageMaker sets the environment variables and how it invokes the script with the three hyper parameters defined in the estimator:

/usr/bin/python mnist_keras_tf.py --batch-size 256 --epochs 20 --learning-rate 0.01 --model_dir s3://sagemaker-eu-west-1-123456789012/sagemaker-tensorflow-scriptmode-2019-05-16-14-11-19-743/model

Because you saved your model in TensorFlow Serving format, Amazon SageMaker can deploy it just like any other TensorFlow model by calling the deploy() API on the estimator. Finally, you can grab some random images from the dataset and predict them with the model you just deployed.

Script mode makes it easy to train and deploy existing TensorFlow code on Amazon SageMaker. Just grab those environment variables, add command line arguments for your hyperparameters, save the model in the right place, and voilà!

Switching to the Apache MXNet backend

As mentioned earlier, Keras also supports MXNet as a backend. Many customers find that it trains faster than TensorFlow, so you may want to give it a shot.

Everything discussed above still applies (script mode, etc.). You only make two changes:

• Use channels_first.
• Save the model in MXNet format, creating an extra file (model-shapes.json) required to load the model for prediction.

For more information, see the mnist_keras_mxnet.py training code for MXNet.

You can find the Amazon SageMaker steps in the notebook. Apache MXNet uses virtually the same process I just reviewed, aside from using the MXNet estimator.

Automatic model tuning on Keras

Automatic model tuning is a technique that helps you find the optimal hyperparameters for your training job, that is, the hyperparameters that maximize validation accuracy.

You have access to this feature by default because you’re using the built-in estimators for TensorFlow and MXNet. For the sake of brevity, I only show you how to use it with Keras-TensorFlow, but the process is identical for Keras-MXNet.

First, define the hyperparameters you’d like to tune, and their ranges. How about all of them? Thanks to script mode, your parameters are passed as command line arguments, allowing you to tune anything.

hyperparameter_ranges = {
    'epochs':        IntegerParameter(20, 100),
    'learning-rate': ContinuousParameter(0.001, 0.1, scaling_type='Logarithmic'), 
    'batch-size':    IntegerParameter(32, 1024),
    'dense-layer':   IntegerParameter(128, 1024),
    'dropout':       ContinuousParameter(0.2, 0.6)
}

When configuring automatic model tuning, define which metric to optimize on. Amazon SageMaker supports predefined metrics that it can read automatically from the training log for built-in algorithms (XGBoost, etc.) and frameworks (TensorFlow, MXNet, etc.). That’s not the case for Keras. Instead, you must tell Amazon SageMaker how to grab your metric from the log with a simple regular expression:

objective_metric_name = 'val_acc'
objective_type = 'Maximize'
metric_definitions = [{'Name': 'val_acc', 'Regex': 'val_acc: ([0-9\.]+)'}]

Then, you define your tuning job, run it, and deploy the best model. No difference here.

Advanced users may insist on using early stopping to avoid overfitting, and they would be right. You can implement this in Keras using a built-in callback (keras.callbacks.EarlyStopping). However, this also creates difficulty in automatic model tuning.

You need Amazon SageMaker to grab the metric for the best epoch, not the last epoch. To overcome this, define a custom callback to log the best validation accuracy. Modify the regular expression accordingly so that Amazon SageMaker can find it in the training log.

For more information, see the 02-fashion-mnist notebook.

Conclusion

I covered a lot of ground in this post. You now know how to:

• Train and deploy Keras models on Amazon SageMaker, using both the TensorFlow and the Apache MXNet built-in environments.
• Use script mode to use your existing Keras code with minimal change.
• Perform automatic model tuning on Keras metrics.

Thank you very much for reading. I hope this was useful. I always appreciate comments and feedback, either here or more directly on Twitter.

About the Author

Julien is the Artificial Intelligence & Machine Learning Evangelist for EMEA. He focuses on helping developers and enterprises bring their ideas to life. In his spare time, he reads the works of JRR Tolkien again and again.

You can use chatbots for automating tasks such as scheduling appointments to improve productivity in enterprise and small business environments. In this blog post, we show how you can build the backend integration for an appointment bot with the calendar software in Microsoft Office 365 Exchange Online. For scheduling appointments, the bot interacts with the end user to find convenient time slots and reserves a slot.

We use the scenario of a retail banking customer booking an appointment using a chatbot powered by Amazon Lex. The bank offers personal banking services and investment banking services and uses Office 365 Exchange Online for email and calendars.

Bank customers interact with the bot using a web browser. Behind the scenes, Amazon Lex uses an AWS Lambda function to connect with the banking agent’s Office 365 calendar. This function looks up the bank agent’s calendar and provides available times to Amazon Lex, so these can be displayed to the end user. After the booking is complete, an invitation is saved on the agent’s Office 365 and the bank customer’s calendar as shown in the following graphic:

The following flowchart describes the scenario:

Architecture

To achieve this automation we use an AWS Lambda function to call Office 365 APIs to fulfill the Amazon Lex intent. The Office 365 secrets are stored securely in AWS Secrets Manager. The bot is integrated with a web application that is hosted on Amazon S3. Amazon Cognito  is used to authorize calls to Amazon Lex services from the web application.

To make it easy to build the solution, we have split it into three stages:

• Stage 1: Create an Office 365 application. In this stage, you create an application in Office 365. The application is necessary to call the Microsoft Graph Calendar APIs for discovering and booking free calendar slots. You need to work with your Azure Active Directory (AAD) admin to complete this stage.
• Stage 2: Create the Amazon Lex bot for booking appointments. In this stage, you create an Amazon Lex bot with necessary intents, utterances, and slots. You also create an AWS Lambda function that calls Office 365 APIs for fulfilling the intent.
• Stage 3: Deploy the bot to a website. After completion of stage 1 and stage 2, you have a fully functional bot that discovers and books Office 365 calendars slots.

Let’s start building the solution.

Stage 1: Create an Office 365 application

Follow these steps to create the Office 365 application. If you don’t have an existing office 365 account for testing, you can use the free trial of Office 365 business premium.

Notes:

1. To complete this stage, you will need to work with your Azure Active Directory administrator.
2. The Office 365 application can be created using Microsoft Azure portal or the Application Registration portal. The following steps uses the Application Registration portal for creating the Office 365 application.

Log in to https://apps.dev.microsoft.com/ with your Office365 credentials and click Add an App.

1. On the Create App Screen, enter the name and choose Create.
2. On the Registration screen, Copy the Application Id and choose Generate New Password in the Application Secrets.
3. In the New password generated pop-up window, save the newly generated password in a secure location. Note that this password will be displayed only once.
4. Click Add Platform and select Web.
5. In the Web section, enter the URL of the web app where the Amazon Lex chatbot will be hosted. For testing purposes, you can also use a URL on your computer, such as http://localhost/myapp/. Keep a note of this URL.
6. In the Microsoft Graph Permissions section, choose Add in Application Permissions sub-section.
7. In the Select Permission pop-up window, select Calendars.ReadWrite permission.
8. Choose Save to create the application.
9. Request your Azure Active Directory (AAD) Administrator to give you the tenant ID for your organization. The AAD tenant ID is available on the Azure portal.
10. Request your AAD Administrator for the user id of the agents whose calendar you wish to book. This information is available on the Azure portal.
11. Admin Consent: Your AAD administrator needs to provide consent to the application to access 365 APIs. This is done by constructing the following URL and granting access explicitly.URL: https://login.microsoftonline.com/{Tenant_Id}/adminconsent?client_id={Application_Id}&state=12345&redirect_uri={Redirect_URL}For the previous parameters substitute suitable values.
    • {AAD Tenant_Id}: AAD Tenant ID from step 9
    • {Application_Id}: Application ID from step 2
    • {Redirect_URL}: Redirect URL from step 5

     Your AAD administrator will be prompted for administrator credentials on clicking the URL. On successful authentication the administrator gives explicit access by clicking Accept.

    Notes:

    1. This step can be done only by the AAD administrator.
    2. The administrator might receive a page not found error after approving the application if the redirect URL specified in step 5 is http://localhost/myapp/ . This is because the approval page redirects to the redirect URL configured. You can ignore this error and proceed
12. To proceed to the next step, a few important parameters need to be saved. Open a text pad and create the following key value pairs. These are the keys that you need to use.
    

    Key	Values/ Details
    Azure Active Directory Id	The AAD Administrator has this information as described in step 9.
    Application Id	The ID of the Office 365 application that you created. Specified in step 2.
    Redirect Uri	The redirect URI specified in step 5.
    Application Password	The Office 365 application password stored in step 3.
    Investment Agent UserId	The user ID of the investment agent from step 10.
    Personal Agent UserId	The User ID of the personal banking agent from step 10.

Stage 2: Create the Amazon Lex bot for booking appointments

In this stage, you create the Amazon Lex bot and the AWS Lambda function and store the application passwords in AWS Secrets Manager. After completing this stage you will have a fully functional bot that is ready for deployment. The code for the lambda function is available here.

This stage is automated using AWS CloudFormation and accomplishes the following tasks:

• Creates an Amazon Lex bot with required intents, utterances, and slots.
• Stores Office 365 secrets in AWS Secrets Manager.
• Deploys the AWS Lambda function.
• Creates AWS Identity and Access Management (IAM) roles necessary for the AWS Lambda function.
• Associates the Lambda function with the Amazon Lex bot.
• Builds the Amazon Lex bot.

Choose the launch stack button to deploy the solution.

On the AWS CloudFormation console, use the data from Step 11 of Stage 1 as parameters to deploy the solution.

The key aspects of the solution are the Amazon Lex bot and the AWS Lambda function used for fulfilment. Let’s dive deep into these components.

Amazon Lex bot

The Amazon Lex bot consist of intents, utterances, and slots. The following image describes them.

AWS Lambda function

The AWS Lambda function gets inputs from Amazon Lex and calls Office 365 APIs to book appointments. The following are the key AWS Lambda functions and methods.

Function: 1 – Get Office 365 bearer token

To call Office 365 APIs, you first need to get the bearer token from Microsoft. The method described in this section gets the bearer token by passing the Office 365 application secrets stored in AWS Secrets Manager.

var reqBody = "client_id=" + ClientId + "&scope=https%3A%2F%2Fgraph.microsoft.com%2F.default&redirect_uri=" + RedirectUri + "&grant_type=client_credentials&client_secret=" + ClientSecret;
    var url = "https://login.microsoftonline.com/" + ADDirectoryId + "/oauth2/v2.0/token";

    Request.post({
        "headers": { "content-type": "application/x-www-form-urlencoded" },
        "url": url,
        "body": reqBody,
    }, (error, response, body) => {
        if (error) {
            return console.log(error);
        }
	 accessToken = JSON.parse(body).access_token;
        if (bookAppointment) {

            BookAppointment(accessToken , //other params);
        }
        else {
            GetDateValues(accessToken , //other params);
        }
    });

Function: 2 – Book calendar slots

This function books a slots in the agent’s calendar. The Graph API called is user/events. As noted earlier, the access token is necessary for all API calls and is passed as a header.

var postUrl = "https://graph.microsoft.com/v1.0/users/" + userId + "/events";
    var endTime = parseInt(time) + 1;

    var pBody = JSON.stringify({
        "subject": "Customer meeting",
        "start": { "dateTime": date + "T" + time + ":00", "timeZone": timeZone },
        "end": { "dateTime": date + "T" + endTime + ":00:00", "timeZone": timeZone }
    });

    Request.post({
        "headers": {
            "Content-type": "application/json",
            "Authorization": "Bearer " + accesstoken
        },
        "url": postUrl,
        "body": pBody
    }, (error, response, postResBody) => {
        if (error) {
            return console.log(error);
        }

        //Return successful message to customer and complete the intent..

You have completed Stage 2, and you have built the bot. It’s now time to test the bot and deploy it on a website. Use the following steps to the test the bot in the Amazon Lex console.

Testing the bot

1. In the Amazon Lex console, choose the MakeAppointment bot, choose Test bot, and then enter Book an appointment.
   
2. Select Personal/ Investment and Choose a Day from the response cards.
3. Specify a time from the list of slots available.
4. Confirm the appointment.
   
5. Go to the outlook calendar of the investment/ personal banking agent to verify that a slot has been booked on the calendar.
   

Congratulations! You have successfully deployed and tested a bot that is able to book appointments in Office 365.

Stage 3: Make the bot available on the web  

Now your bot is ready to be deployed. You can choose to deploy it on a mobile application or on messaging platforms like Facebook, Slack, and Twilio by using these instructions. You can also use this blog that shows you how you can integrate your Amazon Lex bot with a web application. It gives you an AWS CloudFormation template to deploy the web application.

Note: To deploy this in production, use AWS Cognito user pools or use federation to add authentication and authorization to access the website.

Clean up

You can delete the entire CloudFormation stack. Open the AWS CloudFormation console, select the stack, and choose the Delete Stack option on the Actions menu. It will delete all the AWS Lambda functions and secrets stored in AWS Secrets Manager. To delete the bot, go to the Amazon Lex console, select the Make Appointments bot, and then choose Delete on the Actions menu.

Conclusion

This blog post shows you how to build a bot that schedules appointments with Office 365 and deploys it to your website within minutes. This is one of the many ways bots can help you improve productivity and deliver a better customer experience.

About the Author

Rahul Kulkarni is a solutions architect at Amazon Web Services. He works with partners and customers to help them build on AWS